Sunday, September 28, 2014

Lady Gaga and Tony Bennett

Lady Gaga and Tony Bennett are in the news these days.  The reason is that they have just released a new album called "Tony Bennett and Lady Gaga:  Cheek to Cheek".  They are also doing a TV special that will be broadcast in October.  The album and the relationship have become a "thing".  Like lots of other people (that's why it's a "thing") I am fascinated by the fact that it's happening and that it's a "thing".  I try to write about things I think I know something about.  I am pretty ignorant about the music business so I should probably stay away from the subject.  But as something in the way of an experiment I am going to charge ahead anyway.  So now that we have established that I am on shaky ground, in an attempt to at least simulate being on solid ground let me start off the way I start off a lot of these pieces, with some history.

If we go back to the 1800's the music business existed but in a quite different form.  The technology did not exist to record and distribute music.  Theoretically, live performance in a theater was an option but, here too, things did not work very well.  There was no way to amplify sounds so singers had to sing loudly enough and the house had to be small enough so that patrons in the back could hear.  Bands and Orchestras worked better but they were very expensive.  So then as now most music was consumed at home.  Rich people owned a piano.  Poorer people made due with a guitar, banjo, or even a home made flute.  The music business at the time catered to these people by selling them sheet music so they could make music themselves at home.

Then in 1877 Thomas Edison invented the phonograph.  Early versions were not very practical.  For technical reasons Edison used cylinders.  They were big and expensive.  So the first improvement to the technology was the replacement of the cylinder with a thin flat platter.  That helped.  But the way early phonographs worked was that a needle followed a wobbling groove in the record.  The needle was hooked to a diaphragm which was hooked to a horn.  You see examples in movies set in this period.  But the whole process didn't work very well.  There is very little energy in the wobble of the needle.  The energy of the wobble was transformed into sound by the diaphragm and the horn.  So what came out of the horn was pretty quiet.  You had to have your ear pointed at the horn and positioned within a couple of feet of the mouth of the horn to hear anything.  The scenes you see in the movies where several people in a room are listening to one of these old phonographs are fantasy.

Then in 1906 Lee De Forest invented the Triode.  It was basically an incandescent light bulb with a couple of extra parts added in. But the point was that a Triode could be used to make an electronic amplifier.  As with the phonograph, some development was necessary to make electronic amplifiers practical.  But World War I came along to speed things up with the result that usable and inexpensive phonographs incorporating an electronic amplifier became widely available in the '20s.  Now there was a lot more energy available to push air around with so the volume of sound produced by these phonographs made it practical for a group of people to sit around in a room and listen to a phonograph record.  And they did in large numbers.

The triode based amplifier also made radio possible.  But radios are more complex so a practical radio at a price that consumers could afford in large numbers didn't arrive until about a decade later in the '30s.  Radios of the time were quite expensive so a family typically owned one and everyone listened together.  The cementing of the connection between records and the radio happened in 1935.  Benny Goodman was a struggling band leader.  In the early years he worked in New York.  A "network" had just been created so that radio shows could be broadcast across the country.  Goodman was failing in New York.  So he and his band embarked on a cross country tour that finished up in Los Angeles.  Unbeknownst to anyone in New York the band had become extremely popular on the West Coast due to broadcasts of their performances as they crossed the country.  They were greeted by massive crowds when they opened in L.A.

And there is a connection to the modern era.  They played at the Palomar Ballroom.  The event was so popular that they started playing several shows a day.  Live performance was a key component of a band's success at that time and it is again now.  The success of the Goodman band was instantly noticed and the music business quickly reoriented itself to using radio as a promotional tool.  Goodman's success also spun into success for other band leaders like Gene Krupa, Count Basie, the Dorsey brothers, Harry James, Glenn Miller, and others.  This close tie between radio and the music business was to last for decades.

Records and radio continued to evolve in parallel.  The "78" record evolved into the "45" and, with the "33", the "album" (a collection of about 10 "pop" songs).  The album then morphed into the audio "CD", a change in technology but a continuation of the concept.  Radios got better.  They got mush smaller and more efficient with the replacement of the old light bulb based triodes with transistors.  The broadcast specification morphed from "AM" to "FM".  FM produced a much better audio quality and allowed the introduction of "stereo" (two audio channels) to supplant "mono" (a single audio channel).  But something else happened in the '50s.

In the '30s radios were expensive enough so that a family could afford only one.  So people listened to radio programs that had a broad enough appeal to be acceptable to every member of the family.  In the '50s radios were inexpensive enough so that they could be put into cars.  And a family could now afford multiple radios.  This made it possible to split the market.  And thus was born "rock and roll".

I was a kid when this was happening.  And I was confused.  I just couldn't figure out what was and was not "rock and roll".  I spent about a decade wrestling with this conundrum.  I finally decided that "rock and roll" was "whatever they played on 'rock' radio stations".  Enough time has passed so that it has become abundantly clear that I was right.  As an example, the show and movie "Jersey Boys" has put the spotlight back on Frankie Valli, a rock and roll icon of the era.  The original songs may have been arranged to be high energy but the themes are similar to many non-rock "pop" songs. 

Another example:  There is a "country and western" standard that includes the line "somebody done somebody wrong song".  This is supposed to be the core of what makes a C&W song a C&W song.  But the sentiment is a common one in "rock and roll", "blues", "rap", and other supposedly different genres.  I have since decided that this is broadly true.  The boundaries between genres are completely artificial and have much more to do with marketing than with structure and content.  Teens want "not their parents' music" so everybody pretends that "rock and roll' songs are different from "pop" songs.  And that "country" songs are entirely different from both.  But mostly people are people and at some level they all like the same things and dislike the same things.  But people like to have an identity and the type of music they listen to is part of their identity.  So it is important to be able to say with a straight face that "I like country and I hate rock and roll" or whatever.

The music industry figured this out in the '50s and has used this information to advantage ever since.  And, it turns out, (I take my segues where I can find them), one of our two stars first got successful in the '50s.  Back when the theory was that there was only one market that market was called "popular music" or "pop" for short.  (In the U.K. the term "pop" still means "whatever is the most popular" so what we in the U.S. call "rock and roll" the Brit's call "pop".)  But Bennett was firmly in the "pop" camp.  He did not do rock and roll and was not played on rock and roll stations.  Nevertheless, he achieved a substantial measure of success and maintained it for some time.  Another success of the period was Rosemary Clooney.  Today she is remembered mostly as the aunt of George Clooney.  Bennett, Clooney, and many others were part of a large stable of successful performers.  Of that group Bennett is the only one still performing.  Most of them were harmed by rock and roll.  But what did them in completely was the "British Invasion" of the '60s.  The two biggest "Brit" groups were the Beetles and the Stones.  Like Bennett the Stones are the only ones still standing of what was originally a large group.

Bennett was hurt by this double whammy.  By the late '70s it looked like his carrier was over.  But then he did something that turned out to be really smart.  He turned his business affairs and the management of his carrier over to his son Danny.  Bennett had stuck with his "saloon singer" style all along.  His son decided that he should continue to do this.  But he needed to market himself to a new generation.  So Bennett started playing college campuses.  And it turned out there was an audience for Bennett's "old fogey music".  The college kids' parents and grandparents might have been familiar with Bennett's material but the kids weren't.  And the universality of the themes shone through Bennett's "no frills" presentation and connected.  The success of this strategy is best demonstrated by Bennett's appearance on the TV show "MTV Unplugged".  MTV used to be a music operation that pioneered showing music videos on TV.  It was a formula that was easy to imitate so lots of people did.  This hurt the MTV business model so they diversified.

One relatively early effort at this diversification was the "Unplugged" show.  The idea was to have popular "rock" musicians unplug their electronic instruments and play on acoustic instruments instead.  Some of the rockers were actually highly skilled.  After some of them went the "unplugged" route with considerable success the show became popular with both rockers and fans.  Bennett, of course had never gone the electronic route, so unplugged was his natural style of performance.  The show was a risk on both sides but to the surprise of both the Bennett people and the MTV people it was a ratings bonanza.  This catapulted Bennett straight into the "successful crossover artist" category and generated lots of sales for his albums and concert tickets.

And Danny continued to smartly manage Bennett's carrier.  Bennett could perform in the manner he liked to and still be very successful.  As Danny was fixing the public side of the Bennett operation he also fixed the business side.  Old tax and other problems were fixed and the business side was smoothed out so that Tony could focus on performance knowing that the rest of the operation was being run well.  Life became sweet.  And, with Danny's help, Tony became a beloved institution.  The PBS TV show "American Masters" did a show on Bennett in 2007.  The show was produced and hosted by another beloved institution Clint Eastwood.  Another Danny idea was to continue to do crossover events.  So Bennett put out an album called "Duets" in 2006.  For each number Bennett is paired with a different musician.  The list of musicians included James Taylor, Paul McCartney, Elton John, k. d. lang, Stevie Wonder, Tim McGraw, Bono, and others.  The artists were drawn from a broad range of genres.  The album was a success and success breeds imitation so "Duets" was followed by "Duets II" in 2011.  This time one of the artists was Lady Gaga and this is the first time the paths of the two publicly crossed.

But the two had actually met for the first time a short time before.  Bennett attended a charity event that included a performance by Gaga.  He asked to meet her backstage after the show and they hit it off immediately.  This resulted in Gaga appearing on Duet II.  The recording session was documented in a New Yorker story by Gay Talese, of all people.  It would appear that the two have nothing in common.  He's 88 and she is sixty years younger at 28.  He is known for his "straight ahead no frills" approach to performance while she is known for her elaborate productions.  But they are both New Yorkers of Italian descent.  And so are a lot of other people who have never connected.  But the two did.  And the connection is real.  Talese does a good job of putting it on display in the New Yorker piece.

It is founded on a mutual respect for the talent and professionalism each has for the other.  It is often hard to tell from the outside how much of a performance is real and how much is production magic.  I have been aware of the problem for a long time.  I remember listening to rock bands in the '60s.  It wasn't something I was really interested in but I would try to be fair and determine whether they were actually good or not.  I never came up with a good method.  The rule of thumb I finally adopted was "if the band is not drowning out the singer then they must be pretty good".  That's a terrible rule but it was the best I could come up with.  And many years later I found others who use the same pathetic rule I do and for the same reason.  But this rule doesn't work in a lot of cases.

For a while Nancy Sinatra, daughter of Frank, was a "thing".  She sounded pretty good to me so I wondered why her carrier was so short.  I only found out the answer recently.  She sang "You Only Live Twice" for the James Bond movie of the same name.  It turns out they had to splice small snippets from each of the 32 takes she did to get a usable final version.  In other words, she can't sing.  They were able (just barely) to cover this up in the recording studio.  But apparently it was obvious in her live shows.  And at some point everyone decided that the amount of studio work necessary to produce an acceptable final result wasn't worth it.  So with that as background, can Lady Gaga sing?  I don't know but I don't think Bennett suffers fools gladly.  So I take him at his word that she can sing.  And certainly he can sing.  I've seen him sing live.  And the sound guy for that particular show was incompetent.  There's no way any "smoke and mirrors" was going on.

The problem is that there are a lot more people who can sing well than there are successful singers.  Gaga has broken out by adding in a lot of showmanship.  The problem with this is that there is a conventional wisdom.  The conventional wisdom is that showmanship is easy to pull off.  Conventional wisdom couples this with the idea that the only people who use a lot of showmanship are people who have no talent.  And certainly people keep trying to get away with completely faking it (and sometimes succeeding).  Nancy Sinatra was an example at least of the "trying" part.  Some years ago there was a duo called Milli Vanilli.  They were successful for a while.  Then it leaked out that the two performers were just that, performers.  Someone else had actually done the singing while the front men were just lip syncing along in their "live" performances.  And its not always completely black and white whether it's completely real or totally fake.  Andy Williams was a successful singer for many years.  But his "sound" was based on being augmented by a "reverb machine".  It was his electronically reprocessed voice that we liked.  He could and did sing but his actual unaugmented voice just didn't sound that good.  Is he real or a fake?  There is no clear cut answer.  Enough on the "singing" side of things.  Let me now move on to the "performance" side.

I disagree that the performance side is easy.  If it only involves the singing equivalent of "air guitar" where the performance consists solely of going on stage and pretending to sing, as it did with Nancy Sinatra and Milli Vanilli, then the knock is correct.  Most of us are capable of at least a "karaoke quality" level of performance if we don't have to actually sing.  Doing it on stage in front of a cheering mob successfully is harder but it's not that much harder.  But what Lady Gaga does is far more than just standing there and singing.  The whole thing is of a piece with sets, costumes, choreography, etc.  And the parts have to come together coherently into an integrated whole.  She does that extremely well.  Now it's not just her.  There is a whole team contributing.  But I think she sets the tone and makes all the final decisions.  The knock then changes to "well, all she has to do is put together the team".  That sounds easy but it is like putting together a championship professional sports team.  "You just put the team together and get out of the way," right?  If that was that easy then every professional team would win the championship every year.  But they don't.  And, ultimately after the team has done its magic Lady Gaga has to go on stage and make it work for real while everyone is watching.  That's much harder than it looks.

She is the current one and only reigning master of pulling this kind of thing off.  The formula seems obvious.  Go for spectacle, surround yourself with talented people, and just do it.  But there is typically only one person doing it successfully in each era.  So moving backward from the "Gaga" ear let me run through others who have pulled it off.  Gaga has inherited the slot from Madonna.  Gaga has been at the top of her game for from between 5 and ten years, depending on how you measure things.  Madonna was the reigning champion for twenty to twenty-five years.  Both of them built on a foundation of singing.  Neither of them seems to be more than adequate as actors.  But both have developed live shows that are strong on theatrical elements and, as a result, quite entertaining.  But this sort of thing is hard work.  I think what did Madonna in more than anything else is that pulling it off is hard and hard work.  You run out of creative energy and the stamina to do the demanding work the show requires.  Gaga at 28 is in her prime.  I am sure she would like to be able to stay on top of her game as long as Madonna did.  And Madonna has never completely faded from view.

Before Madonna the person who pulled the required mixture of skill and theater off best was Liberace.  He has mostly faded from the scene but at his peak he was the most successful entertainer to date.  And his story is more surprising by far than that of Madonna or Gaga.  Both of them came from modest roots but they were trying to do something (be successful singers) that a lot of people before them had succeeded at so there was an understanding of how to go about it.  Liberace started out as a classical concert pianist.  You may recognize the name Van Cliburn or you may not.  Cliburn is the most famous concert pianist ever and it's a toss up if people now recognize the name.  That's what Liberace was up against. He decided he really wanted to be famous and successful and the opportunities available to even the top man in his field just weren't enough.  So he began reinventing himself.

He became flamboyant.  He dressed in crazy outfits.  He started putting elaborate candelabras on his pianos.  But most importantly he started interacting with his audiences.  He was very good at being charming. He parlayed all this into a successful TV show.  This in turn fed into successful concert tours in both the U.S. and Europe.  As his popularity and fame increased he became more flamboyant.  And, in parallel with Hugh Hefner of Playboy Magazine fame, and before the eponymous TV show started airing, Librace became famous for living a "lifestyle of the rich and famous".  He extravagant lifestyle became part of the act.  Through it all he maintained a close connection with his audience and was able to keep his popularity high.

This was all in spite of the fact that he was obviously gay in an era when you weren't allowed to be gay.  Everybody liked him so everybody pretended he wasn't gay in spite of the fact that his whole persona screamed "drag queen" before the term was even in wide use.  He even successfully sued two magazines for just hinting he was gay.  He was still going strong when AIDS caught up with him in 1987.  The Liberace Museum was a popular mainstay of Las Vegas until a few years ago.  Liberace was very successful at mixing skill, in his case piano playing, with showmanship.  And to do that in a very gay way in an era when being gay got you thrown in jail is a tremendous accomplishment.

And I want to go back one more generation to a person who, like Liberace, is only dimly remembered now.  That was another woman.  Her name was Mae West.  Here again we have a mix of talent  and showmanship.  Ostensibly Mae was a singer and actor.  But her singing was limited and very stylized and not that dissimilar to just speaking.  And when she acted she always played Mae West.  So if she was so apparently lacking in talent why am I putting her on this list.  Actually she was tremendously talented.  It was just that her talent was not singing or acting.  What she was talented at was as a comedienne.  She was brilliant at conceiving and executing comedy.  She was born in 1893 and had her greatest success in the late '20s and early '30s.  She had a great deal of success in vaudeville and successfully translated her act to the movies.  She played a vamp and specialized in double entendres.  A famous one is "is that a pickle in your pocket or are you just happy to see me".  Another is "When I'm good I'm very good.  But when I'm bad I'm better."

West was too old and not pretty enough to be a convincing "hot chick" but people went along with the gag because her movies were fun and funny.  She conceived of situations into which she could introduce great one-liners that people loved.  What West has in common with the others is the over the top character she invented for herself.  I think the character Liberace invented for himself was that of a male Mae West.  West was eventually done in by the Hollywood system.  The studios were pressured into putting in a very intrusive censorship system.  A great deal of violence was tolerated but any hint of sex was not allowed.  During this period the guy always had to have one foot on the floor in bedroom scenes, couples had to sleep in separate twin beds, and bathrooms had no toilets in them.  Even though all of West's gags involved inference rather than anything explicit the censors cut them out.  Without the good gags the movies had nothing left.

So what is the connection between our two stars?  In Bennett's case it might be as simple as he respecting her talent.  If you look at the range of talent that he has teamed up with in his two "Duet" albums it is awesome.  They span the gamut of musical genres.  But every one of them has a reputation for first class musicianship.  If all these other people are so skilled then if Lady Gaga is one of them it must be because she is at the same level of of musicianship.  As I said, this is something I do not have an informed opinion on.

It might also be said that Lady Gaga represents an opportunity for Bennett to expand his audience base.  But he got that benefit by partnering with her on their "Duets II" song.  Bennett has done other joint events with other performers from "Duets" and "Duets II" but it hasn't happened often.  He does not seem to have a significant non-professional relationship with any of his other partners.  And he hasn't tried to put together a whole duet album with any of them.  But he has with Lady Gaga.  So, although it might be beneficial for him to continue to get in front of her fans, I don't think it drives the personal relationship between the two of them.

So what about Lady Gaga?  What's in it for her?  Again, at bottom, she might just respect Bennett's musical ability.  She has said so on numerous occasions.  But beyond that, what else?  Well, Bennett is respected as a person who genuinely knows how to sing well.  By validating Lady Gaga as another person who knows how to sing well some of the respect accorded to Bennett rubs off on to Gaga.  It's nice to be famous and to make a lot of money.  But it is also nice to be respected as someone with genuine talent and skill.  Bennett seems happy to validate her.  And he seems quite sincere when he does it.  So there might just be a mutual admiration society thing going on.  That's nice and it could be a sufficient base to support their relationship.  But I think there is one more thing.

Gay Talese brought this out in his piece in New Yorker.  During the interviews for the piece Bennett spoke about Amy Winehouse.  Winehouse was one of the other "Duet" artists.  Winehouse had committed suicide shortly before the interviews.  She was on Bennett's mind because he had been genuinely concerned that she might take her life.  He wanted to prevent that if he could.  As a long time show business survivor he felt he might be able to get through to her.  He was not able to get in touch so nothing came of it.  And I'm not trying to imply that Lady Gaga was or is suicidal.

But Lady Gaga had gone from nobody to superstar in very little time.  That is a very hard transition to make.  There was talk that for a while that Lady Gaga was thinking about getting out of the business.  With her Bennett saw an opportunity to do some mentoring, to be the show business equivalent of a father figure.  And that relationship is apparently now present. Bennett has been able to show her an example of someone who has been in show business a long time and has found a way to survive and thrive.  Beyond that he has been able to give her some specific advice for handling various pressures and situations that she has apparently found helpful.  So Bennett has been able to engage in some "passing the torch to the new generation" and Gaga has been able to get some practical advice that has helped a lot.  Lady Gaga herself credits Bennett with reviving her interest in staying in show business.

The new album is full of Bennett stuff.  But apparently Lady Gaga is quite happy.  It's good material that has stood the test of time.  She is wise to recognize that.  And, as an artist, what's not to like about an opportunity to do good material?  The TV special is scheduled to air on PBS on October 24.  And this is a case where Lady Gaga stands to possibly pick up some Bennett fans rather than the other way around.  I hope she does.

Friday, September 12, 2014

The Great Internet Disaster

You remember the Great Internet Disaster.  You don't?  Well, I do!  No actually I don't for the good and substantial reason that it didn't happen.  But pretending it did for a moment makes for a much better title.

This is actually a "good news" story, a story about what didn't happen.  But, with the exception of a few religious types, people who are trying to attract and keep your attention avoid good news like the plague.  And, while we are on the subject of plague, have your heard the latest about Ebola?  See!  It works.  I have now thrown in two phony disasters and you are paying attention.  The one in the title is completely phony.  The other one is real but it only affects those people way over there in Africa.  We don't care about them so the media has to pretend that it is about to strike our shores any second now.  All right.  Enough with the phony stuff.  I promise the rest of this post consists of 100% actually true stuff.

So what is "The Great Internet Disaster"?  Well, what if you tried to connect to your favorite web site and you couldn't?  What if you couldn't connect to anything?  And I'm not talking about some kind of temporary outage.  I am talking about forever.  Wouldn't that be a disaster?  It would.  And it could have happened.  Officially, it could still happen but I think it never will.  But a lot of people have spent a lot of sleepless nights worrying about it.  And people have been tracking the problem for almost two decades now.  And the thing the people worried about and tracked actually happened.  So what is "it"?

It's technical, not complicated but technical.  The Internet started out many decades ago as the ARPANET.  It was a research project sponsored by the Federal Government.  Back in the dark ages it was possible to connect computers together.  But there was no agreed on standard way to do this.  Each company came up with their own scheme.  Then they took out patents and copyrighted stuff and kept trade secrets and did other things, all to make sure some other company couldn't use whatever they came up with.  So you had all these different methods.

Then you had IBM.  IBM didn't have one method.  They had three or maybe more but at least three.  They even had an acronym for each.  Then had BTAM, and TCAM, and VTAM.  Ok.  All the acronyms had four letters and all of them ended with "AM", which stood for Access Method, but that was pretty much it.  Even though all three methods were developed by IBM for use on IBM equipment they didn't talk to each other.  So a large company could find itself in a situation where some of its IBM equipment couldn't talk to other IBM equipment even though it was all made by IBM and all owned by the same company.  So getting an RCA computer owned by one company to talk to a GE computer owned by another company was pretty much impossible.  And yes, both RCA and GE used to custom design and build computers that they sold to other companies, kind of like Apple does today.  Anyhow, this whole business of getting computers to talk to each other was nearly impossible then.  But the government, or at least the military part of the government, thought it was important to figure out how to get all kinds of different computers to talk to each other.  So they funded the ARPANET project.

Computers are basically really fast math machines.  At bottom all they know is numbers.  They can handle text and pictures and lots of other things now.  But the trick is to turn everything into numbers.  So there is a number that means "A" and there is a number that means a specific shade of Yellow, and so on.  So it stood to reason that one of the things you want to do if you are trying to get computers to talk to each other is to assign each computer a number.  Then computer number 17 could send a message to computer number 37 and off we go.  At the time the ARPANET project was active each computer cost millions of dollars.  So the way the ARPANET people set things up they allowed for as many as 64 computers to be hooked up to their network.  And it worked.  The ARPANET project got a bunch of computers to talk to each other.  And it almost immediately became a victim of its own success.  Everyone wanted to hook their compute up to the ARPANET.  And when more than 50 computers were hooked together and there was still a long list of other computers people wanted to add it was time for the ARAPNET equivalent of "Huston, we have a problem".

The number "40" appears in the Bible a lot.  Have you ever wondered why?  Well, back 2,000 years ago 40 was a really big number.  Most people did arithmetic by counting on their fingers.  So if, for instance, you had a lot of sheep after a certain point it became very hard to keep track of exactly how many you had.  The cutoff for how many was "so many it's hard to count them all" was more than 10.  And I'm sure it varied from person to person.  But 40 was definitely too many for almost everyone.  So 40 was used for "so many it's practically the same as infinitely many".  So whenever anyone wants to say the ancient equivalent of a gazillion they say 40.  In the initial design of the ARPANET 64 seemed like a gazillion.  But it obviously wasn't.  So the ARPANET people started working on ARPANET v2.

Specifications, as we now know, evolve and keep evolving.  We are now used to the iPhone 6 and Windows 9 and so on.  So it's no surprise that the specification for this computer number thing have gone through a number of versions.  But for a long time it has been something called IPv4.  Now the ARPANET people really wanted to fix the whole "size of the computer number" once and for all.  But they had to pick some specific size because that's how computers work.  So they looked around for a number that, while not actually infinite, was so big that the problem would be permanently fixed.  They looked for the practical equivalent of a gazillion.  And they settled on a number slightly larger than 4 billion.  They made the size of the field for the computer number 32 bits long.  And that allowed for over 4 billion different numbers.  Now when they did this computers were fantastically expensive.  So 4 billion really did sound like a gazillion.  And everything was fine until the middle of the '90s.

When Microsoft was designing what eventually became Windows 95 it seemed like a good idea to include the capability for connecting computers to servers using phone lines and an interface device called a modem.  So Microsoft put in a bunch of features to support this.  And initially that sounded like it would get the job done.  But a few years earlier a Senator named Al Gore had sheparded a piece of legislation through congress and into law that turned the grandson of the ARPANET, something called NSFNET, into the Internet.  This was the final step in an evolution from a government sponsored network for connecting a few military computers together to a public network that anybody could connect to.  And it didn't matter what brand of computer you had.  A lot of money and talent had been invested over the years in figuring out how to hook pretty much any model of any brand of computer to what was now the Internet.  And people started asking "can I hook my Windows 95 computer to the Internet?"  Microsoft did some quick work and by the time Windows 95 debuted in August of that year the answer was "yes".  And, of course, once Microsoft did it then Apple and everyone else had to do it too.  So within a couple of years every new PC sold had the ability to connect to the Internet.

Back when computers cost millions of dollars 4 billion did seem like a bazillion, way larger than the number of computers anyone could imagine would be built.  And things were still ok when Windows 95 came out.  Computers still cost thousands of dollars and most of the world couldn't afford them.  But it started people worrying.  And there was another problem.  You couldn't practically use all 4 billion of those addresses.

To manage the administration of these addresses, now called "IP Addresses", the people who ran the Internet (First the Department of Defense, then the National Science Foundation, finally a group of non-profits) set up a process where an organization could get control of a "subnet", a set of IP addresses.  These subnets came in 5 classes:  "A", "B", "C", "D", and "E".  If you filed the proper paperwork and it was approved you were given what amounted to ownership of a specific subnet of the appropriate class.  Once you were granted control you could do whatever you wanted with each specific address in your subnet.  That way the administrative group only needed to keep track of the classes.  And this whole system worked fine for decades.  So what was the system?

Let me start with the weird classes.  Class "E" was for research.  Class "D" was used for broadcast.  Neither of these classes is used much.  But there are about a half a billion IP addresses reserved for each.  So between the two classes about a billion IP Addresses are off limits.  The classes everyone used were "A", "B" and "C".  They were the same except for size.  A class A subnet is about 16 million addresses (2 to the 24th power).  A class B subnet is about 65,000 addresses (2 to the 16th power).  A class C subnet is exactly 256 addresses (2 to the 8th power).  The idea is giant companies or organizations would get a class A.  IBM and AT&T each have one.  The Federal Government has several.  Medium sized companies and organizations would get a class B.  In some cases they were allowed to get several.  At one time Microsoft controlled several class B subnets.  Finally, of course, small companies and organizations would get one or more class C subnets.  The company I used to work for controls 5 class C subnets.  In round numbers, the space reserved for all class A subnets constituted about 2 billion IP addresses.  The space reserved for all class B addresses constituted about a half a billion addresses.  The space reserved for all class C subnets constituted the remaining half billion IP addresses.

Up until the late '90s everything was fine.  But by this time a double whammy was becoming apparent.  Transitioning from a semi-open network run by the U.S. Government to a fully open network, the transition from NSFNET to the Internet, caused the list of people, companies, and organizations interested in connecting to the Internet to skyrocket.  Added on top was the fact that a relatively inexpensive "network card" could be added to a relatively inexpensive "PC" meant that it was financially possible to add a lot more computers to the Internet than it had been at any time in the past.  So Internet connections skyrocketed and the rate at which IP addresses were being consumed also skyrocketed.  This was a cause for concern.  But the fix was relatively simple.

For most organizations a class C was fine.  But there turned out to be a lot of examples where it really did not work.  My company with 5 class Cs was ok.  That was a manageable number.  But what about 10 or 20 or 50?  That got unwieldy.  The obvious solution was to get a class B instead.  But this was the equivalent of 256 class Cs.  If an entity actually needed 50 class Cs but got a class B then 80% of a class B would be wasted.  The same was true of class Bs.  For some companies (Internet Service Providers - ISPs for short) one or a few class Bs were unwieldy.  So they would go for a class A.  But again a class A was 256 times bigger than a class B.  So if you actually needed 50 Bs 80% of the A would get wasted.  And frankly an A was a prestige symbol.  So a number of big companies got an A just because they could.  Ford Motor Company owns an A.  This probably makes sense.  It's a big company and it is spread all over the world.  But Halliburton owns one too.  I find it hard to believe that Halliburton couldn't get by with a few Bs.  And then there's big Pharma.  Eli Lilly has one as does Merck.  I think several companies got an A became they had enough political, financial, and technical clout to get one not because they really needed it.  So the system was being abused.

The fix was to offer more options.  Instead of the simple A-B-C system a system called CIDR was introduced to replace it.  A bunch of subnet sizes that were smaller than an A but bigger than a C were now available.  This meant that someone who needed 10 Cs could get a "CIDR block" that was equivalent in size to 16 Cs.  (It's that whole power of 2 thing).  A company that needed 20 Bs could get a CIDR block that was equivalent in size to 32 Bs.  This immediately took a lot of pressure off the system.  Companies and organizations could get a CIDR block that was reasonably close to the number of actual IP addresses they needed.  So the rate at which IP addresses were eaten plunged.  Problem solved.

And it was for a while.  An "Internet ready" PC was still fairly expensive.  And the way most people at home connected up was by dialing in to AOL or some other ISP.  AOL could pool IP addresses.  It could give an IP address to you temporarily while you were connected.  But it could take it back when you hung up and give it to someone else who dialed in later.  So the pressure from home use was manageable.  Work PCs were still a problem.  But the introduction of CIDR took a lot of the pressure off here too.  Things looked pretty good until smart phones came along.  When smart phones came along it became possible to put a browser on the phone.  It was a selling point so the mobile companies did it.  I have a crap cell phone and even it has a browser on it.  It doesn't really work but it would if I had a better phone.  And smart phones are cheap enough so that they are not just popular in the U.S. or maybe Europe.  They are popular all over the world.

The mobile carriers use the same trick the ISPs use of sharing IP addresses.  But there are 7 billion people in the world.  And about 3 billion of them own a mobile phone.  Most of those phones are not currently web capable but they soon will be.  The advent of smart phones cranked the rate of IP address consumption back into the stratosphere.  And people started seriously worrying about the problem.  People had already started worrying.  Serious stories predicting that we would eventually run out of IP addresses started appearing in technical journals in the '90s.  One of the first credible predictions of when we would actually run out was made in 2003.  The prediction was that we were good for ten to twenty years  But another prediction made in 2005 said we only had "4 to 5 years".  What happened in the mean time was that people could see where smart phones were going.  In 2010, 5 years after the prediction, we were still ok?  What happened?

CIDR was a "lifetime extender".  It worked.  And a second lifetime extender called NAT was introduced.  NAT is IP address sharing technology.  I have a home network with several computers on it.  All of them access the Internet.  I also have some smart devices like my TiVo DVR and my Blu-Ray DVD player, both for my TV.  Both of them need an Internet connection some of the time.  But Comcast, my ISP, only allocates one IP address to my house.  The details are complicated but the bottom line is that NAT works.  All of the "Internet capable" devices in my house can access the Internet because NAT allows my one IP address to effectively be shared by all of them.  In fact, hundreds of computers can use NAT to share one IP address for internet access.  Everything is kept straight and each computer can access what it needs without interfering with any of the others.

At one time my old company was using about half the 1,200+ IP addresses the 5 class Cs represented.  Now the same company has lots more machines, each of which has its own separate IP address.  The NAT setup allows hundreds of machines located on the internal corporate network to independently address hundreds of servers scattered across the Internet.  If pressed, my old company could now get along just fine with a small part of a single class C.

So CIDR and NAT were lifetime extenders.  But ultimately they both failed.  The Internet has run out of IPv4 addresses.  The event was officially commemorated on February 2, 2011.  For three and a half years (as I write this) we have been out of IPv4 addresses.  That should have resulted in the Great Internet Disaster.  New servers are put up each day.  Without IP addresses for these servers no one would be able to connect with them.  New PCs (not as many as in the old days but still lots) get hooked up to the Internet each day.  There should be no IP address to give them so they should not be able to connect to anything.  So what happened?  Am I conning you about this whole "we are out of IP addresses" thing.  Remember, I promised I would tell the truth after the part where I got your attention.  And I am.  We are officially out of IPv4 addresses and have been for years.  I also promised that the Great Internet Disaster was never going to happen.  So let move on to the permanent fix next.

As I indicated above, CIDR and NAT are extenders.  They are not the permanent fix.  And it should not be hard to figure out what the permanent fix should be:  Make the number bigger.  That's what they did before.  They changed the number of computers that could hook up to the network from 64 to 4 billion.  How about doing the same thing again?  There was some back and forth on the details but that is exactly what was done.  There is now a new thing called IPv6.  (Something that got called IPv5 came and went without much of anyone noticing so they had to tick the "v" number up to 6.)  Again,the idea was to come up with a gazillion-like number, a number so big that it would effectively act like infinity.  Ipv4 addresses require 32 bits.  They went with four times as many bits for IPv6.  The new "gazillion" is 4 with 38 zeroes after it.  That's enough addresses to individually label every subatomic particle in every atom on the surface of the earth.  In the same way that going from 6 bits (64) to 32 bits (4 billion) seemed like going to a number much larger than anyone could imagine using up the hope is that by going from 32 bits to 128 bits the same "more than anyone can imagine using" effect will be achieved.  The specifications for IPv6 were developed in 1995 and fine tuned over the next couple of years.  There is a general consensus that this is big enough.  We have had the fix in hand since 1995 (or 1998, if you want to wait for all the tweaks to be completed).   So why am I talking about a "problem" that supposedly has been fixed for more than a decade?

Well, there's the whole "transition" problem.  When ARPANET was moving from a 6 bit address to a 32 bit address there were less than a hundred computers involved.  These were all located in universities with capable Computer Science departments or at government research establishments or the like that were stuffed with computer whizzes.  They all just set to work updating everything to conform to the new rules.  It didn't take long to get everything cut over.  Remember, the ARPANET was a "research and development" project at that time.  It was annoying if some part of it stopped working for a day or so but it was not the end of the world.  The Internet of today is a whole different kind of animal.  You just don't take the whole thing or even large parts of it for a couple of days so you can upgrade something.  That makes it harder to roll something like IPv6 out.

Let's start with the "not a problem" items, items that you might think would cause problems but actually didn't.  Changing from IPv4 to IPv6 required no hardware changes.  The new stuff would work just fine on the old hardware.  In fact, you could run IPv4 messages and IPv6 messages through the same components at the same time.  One type of traffic would be invisible to the other but both would work just fine.  The next issue was plumbing.  For instance there is a system called DNS.  It's the system that turns a computer name like WWW.WIKIPEDIA.COM into a number like 198.35.26.96 (the standard way of representing an IPv4 address as numbers).  Technical changes were made to how DNS was supposed to work and updates were made to DNS software.  Once that was done then the DNS system could handle any combination of IPv4 and IPv6 requests.  Similar technical changes and software changes were made to the other parts of the Internet.  Everything was supposed to be interoperable so making the changes to support IPv6 didn't stop IPv4 from working.  And not everybody needed to make their changes at the same time.  It was just necessary that everyone make the changes.  And they did.  Everything has been in place to support IPv6 for years now.

Then there was the "small problem" issue.  Software changes needed to be made to computers so they could deal with IPv6.  This was harder but not impossible.  All this networking stuff goes through some software called a "protocol stack".  Again its complicated but again the technical details are not important to us.  The best solution in most cases was to go to a dual stack.  One stack would handle the IPv4 stuff and the other the IPv6.  The routing through the right stack could be done automatically by the stack software.  So vendors like Microsoft had to write some more software for the new stack but that was it.  Not all old systems can handle IPv6.  But if you bought a new PC any time in the last few years everything is built in and ready to go.  There is still a lot of old systems that can't handle IPv6 but everything else does.  So Apple and Microsoft and the Linux people needed to do some work.  But they finished that up several years ago too.

That leaves the big problem.  Let's say you have a fancy new computer that is all set up to handle IPv6.  But you want to access a web site that is IPv4.  What's the problem?  Since the Iv6 address is so much bigger you just carve out a slice and say "if I put this IPv6 address in I really mean this other IPv4 address".  That's easy to do and the message you send off should get to where it is supposed to with no problem.  But communication to a web site is bidirectional.  You send a request to the web site.  It sends back a response (the thing that ends up on your screen when you click on the button).  How does this get back to your computer?  That's the problem.  There are way more IPv6 addresses (that's the point) so you can't just do the reverse of the translate thing we did to get from the v6 address to the v4 address.

People have come up with a bunch of ways to handle this problem.  But none of them work very well.  If you are a web site you can just have an IPv4 address and an IPv6 address for your site.  That means you can support requests from both IPv4 and IPv6 computers.  In fact, when I looked up the Wikipedia IP address I got two answers.  One was an IPv4 answer and the other was an IPv6 answer.  But the problem is that when you work through all the permutations what you end up with is giving everything an IPv4 address and an IPv6 address.  That doesn't solve the problem.  If you are going to do that you might as well just stick with IPv4 addresses exclusively.

Before doing the final "reveal" let me cover one more "extender" solution.  The Internet started out as a government funded project.  When it went public it went non-profit.  None of the institutions that were put in place to administer the modern Internet are for-profit companies.  There is also a strong "public interest" mind set that still runs through these institutions.  So the idea that something like an IP address should be a valuable asset that can be bought and sold went against the grain.  So none of these institutions wanted to officially get into the business of buying and selling IP addresses.  But a few years ago they gave in and unofficially sanctioned the trading of ownership of blocks of IP addresses.  The reason we haven't actually run out of IPv4 addresses in spite of the fact that we have officially run out of IPv4 addresses is that you can now sell your unused IPv4 addresses.

Remember those companies I talked about that own class A subnets.  Let's say that you are company "X" that owns a class A.  It's a hassle to figure out if you are using it.  You just might be using parts of it somewhere.  But it would take effort to find this out and effort translates into costing the company money.  So the safest thing to do is to hang on to the A and pretend you are using it for something important.  But let's say you can sell all or part of the class A for money.  Now it is worth while to check around.  So that's what's happening.  Companies that own class As or even class Bs have been quietly checking around.  In lots of cases they find they can part with all or most of the IP addresses in the subnet pretty easily.  So they do for a fee.  These unofficial sales are keeping things alive.  But this is just another of those extenders.  It is not a permanent solution.  So what is?

It turns out that those same smart phones that caused the latest wave of problems have opened the path to a permanent solution.  If you go to the Apple store you will find hundreds of thousands of applications.  Are all these applications actually unique?  Not really.  On a PC if you want to work with a business you connect to its web site.  On an iPhone (or any other smart phone) you click on the appropriate application.  The "app" has replaced the web site.  If you click on the button for the app and the right stuff comes up do you really care how the connection was made?  No!  So if an app uses IPv6 to connect to the server of whatever company you want to deal you don't are as long as it works.  The "G4" specification for mobile phone networks specifies IPv6.  Companies configure their servers for IPv6.  Then they build the app you download from the "app store" for IPv6 and you don't know the difference.  Over time all the traffic associated with mobile phones will migrate automatically to IPv6.  That relieves the pressure on IPv4 two ways.  First there is no need for IPv4 addresses to support mobile traffic for these newer devices.  But second, the wide adoption of mobile phones is causing a decrease in the use of PCs (and tablets, a PC for our purposes).  So the pressure to come up with more and more IPv4 addresses diminishes.  At some point not that many years down the road the number of IPv4 addresses we need will actually go down.  When that happens we can officially call the "disaster" I have outlined solved.

Finally, let me broaden my focus quite a bit.  Let me summarize the issue under discussion in the broadest terms.  There is a situation.  The people who understand the most about what is going on start worrying.  At some point they start making their concerns public.  Their next step is to say "something must be done and soon".  They come up with a bunch of "solutions" but these solutions are not adopted, in part because they would be disruptive.  The experts say "you MUST do something about this problem right away or really bad things will happen".  The solutions the experts came up with are still not implemented.  The bad thing doesn't happen.  So maybe we should never have listened to the experts.

That is a lesson that could be applied to the "IPv4 Address Exhaustion Problem", as what I have been discussing is technically called.  But it is the wrong lesson.  What the experts said all along was right.  And the experts did come up with the right solution, namely IPv6.  The only thing the experts got wrong was the details of how we will pull off the migration from IPv4 to IPv6.  The iPhone, the poster child for smart phones, only dates back to 2007.  During most of the time the experts were studying the problem smart phones did not exist in any significant numbers.  Also, Apple for its own reasons came up with the first successful computer "app store".  And Apple pushed for the replacement of the browser/server combination that worked very well on PCs with the app/server model.  They did this not for any grand reason.  They did it so they could more tightly control the Apple ecosystem so they could make more money.  It is only an unintended side effect that the app/server model turned out to be what makes possible a seamless migration from IPv4 to IPv6.  And these very same experts have been ready, willing, and able to facilitate this new path, a path not of their creation, at every turn.  They are interested in solving the problem, not attaining great glory or gaining wealth beyond your wildest dreams.

The first summary also fits another problem facing us, a much bigger problem than IPv4.  That problem is greenhouse gasses.  A lot of people want very much for people to ignore the experts.  So they push the rest of the original scenario.  We are supposed to believe that if we do nothing then the bad things will never come to pass.  But here too the experts should be listened to.  In the IPv4 case Apple did something in a successful effort to make a lot of money.  And the thing they did turned out to be very helpful.  In the greenhouse gas case there are a number of companies and individuals that are doing things in order to make a lot of money.  Like Apple they are succeeding, and to a massively greater extent than Apple.  But unlike with Apple, these people are not being helpful.  They are deliberately making things worse.

In the IPv4 case there really was no one arguing that there was no problem.  There really is no one in the scientific community who has seriously studied the problem of greenhouse gasses and who is not being paid to say so, that says there is no problem.  In the IPv4 case there were a lot of people that worried about the expense and disruption involved in fixing it.  Before the "mobile fix" showed up there was ample evidence that these concerns were legitimate.  In the early days before the mobile fix appeared the migration did not go well.  In the case of greenhouse gasses there are a lot of people that also are concerned about the expense and disruption of a fix.  It is a very legitimate concern.  In the IPv4 case it was pretty simple to figure out what needed to be done.  It was just hard to figure out how to pull the migration to IPv6 off.  What the mobile fix did was to effectively sidestep the problem.  Non-mobile devices would never be migrated.  The greenhouse gas problem is much more complex and much more difficult.  At its most basic level the fix could simply consist of pulling all that excess Carbon Dioxide out of the air or never putting it in the air in the first place, in effect sidestepping the problem.  But nobody really knows how to do that.  Other amelioration or mitigation ideas, the equivalent of migrating non-mobile devices to IPv6, are even harder to figure out how to pull off.  The greenhouse gas problem is also a much more slowly moving problem than the IPv4 problem.  Back in 1990 when it was laid out in the IPCC report no one was seriously concerned about running out of IPv4 addresses.  15 years later we are, if anything, going backwards on greenhouse gases.  But we are chugging along quietly with the solution to the IPv4 problem.

Ultimately, a permanent solution for the IPv4 problem kind of just fell out of the sky.  But remember that before this happened a number of "extender" solutions were developed and implemented.  They kept things going long enough for the permanent solution to appear.  And when the permanent fix appeared it was identified and embraced.  So none of the actions of the experts were wasted.  It was important to pay attention to them all along the way.  We wouldn't have made it without them.

Wednesday, September 3, 2014

ISIS - Do Something Stupid Now

The "do something stupid" part of the title is a steal from a letter to the editor I wrote recently.  Here's a link to it:  http://blogs.seattletimes.com/northwestvoices/2014/06/14/iraq-wait-dont-do-something-stupid/.  A lot of people have weighed in on ISIS/ISIL since it became a hot topic back in June.  I accused them of being stupid then and unfortunately they have remained stuck in the same rut.  The details have changed a little.  But most bloviating on the subject still translates to "President Obama is wrong, wrong, wrong".  In spite of the many mistakes Obama has supposedly made things are actually somewhat improved, not than anyone has noticed.  So let's review.

In June ISIS (there was no argument at the time as to what it should be called) had just taken control of a large chunk of Iraq, mostly in Anbar province.  "They are unstoppable" was taken as gospel at the time.  The countdown clocks were started for how long it was going to take them to sweep in and take control of Baghdad.  Would it be weeks or only days?  But since then ISIS has in fact been stopped.  The Baghdad offensive never materialized.  ISIS has racked up some gains but it has suffered more losses.  In fact, the conventional wisdom now has it that they are on the defensive.

Between then and now we have had the Yazidi rescue.  The Yazidis are a religious minority that lived in an area that ISIS was able to gain control of.  A large number of them fled to Mt. Sinjar.  All the initial reporting and analysis indicated that they were doomed and that the world was in for another instance of genocide.  But the story has a happy ending.  The Yazidis were rescued in a complex coordinated effort.  There has been exactly zero praise of the President for pulling this off.  And so it goes.

The problem is that the media likes "bang bang".  Gun camera footage of a missile strike is very popular and is played endlessly on TV.  There were some missile strikes in the Yazidi rescue so that's what the media has focused on.  They were helpful but by themselves they would have been completely ineffective.  Let me focus on the immediate Yazidi rescue and then pull back and take a  look at the broader situation.

At the beginning of the incident, and by this I mean the beginning of the U.S. public's introduction to the incident, you had 50,000 Yazidis stuck on Mt. Sinjar.  They were surrounded by bloodthirsty and unstoppable ISIS troops and their complete destruction was just days away.  So what happened?  Well, the first thing that happened is that the U.S. launched some airstrikes from drones and manned aircraft.  This took out some ISIS artillery and put ISIS troops on notice that they would not be allowed a free hand.

Next, the U.S. airdropped some supplies -- food, water, medicine.  It wasn't enough to take care of 50,000 people but it put the Yazidis on notice that someone was paying attention.  The airdrops and missile attacks continued.  A rescue mission was flown in and a small number of Yazidis were evacuated.  Shortly thereafter Kurdish troops were able to move into the area and make contact with the Yazidis.  The Kurds opened a safe corridor along which the Yazidis were able to evacuate.  Everyone who wanted to leave the mountain was able to do so.  Finally, the number of Yazidis turned out to be substantially less than 50,000.  I believe that there are still people living on Mt. Sinjar but they are the people who have always lived there.  I also believe that ISIS has now lost interest in Mt. Sinjar.  That sounds like a big win for the good guys.  And the President can legitimately be given credit for a good chunk of this good news.  But the silence, as they say, has been deafening.  Now let's pull back and look at a wider context.  I am going to start by only pulling back far enough to look at Iraq.

So how did we, and by "we" I actually mean Iraq, get into this mess?  Iraq is a fantasy country.  It was brought into existence in the 1920s as the result of the collapse of the Ottomon empire and the machinations of the British and the French.  It is a mash up of a bunch of political, religious, and ethnic groups that suited the British and the French at the time but did not take into account any facts on the ground.  So it has no natural coherence.  That doesn't mean it can't be successful anyhow.  It hung together reasonably well until 2003 when we invaded it.  But it mostly hung together due to iron fisted domination by a small group of people at the top.  The last in this line was Saddam Hussein.  When the U.S. invaded they toppled the Hussein government and Iraq has been spiraling out of control since.  All the tensions Hussein had been keeping bottled up now were now able to escape.

And you had outside forces aiding and abetting this.  The Iranians favored the Shiites.  Saudi Arabia and others favored the Sunnis.  The long oppressed Kurdish minority saw an opportunity to finally gain some autonomy.  The U.S. acting from outside tried to exert some force to counteract this tendency to fragment.  But the Iraqis saw their chance to get control of their own affairs and took it.  Al Maliki was then able to operate independently.  He implemented policies favoring Shiites and Iran and tried (fairly successfully) to weaken his opponents.  The Kurds were able to fend him off to a considerable extent but the Sunnis, particularly those in Anbar province, weren't.

ISIS is at bottom a Sunni movement.  As such it has received a considerable amount of support from the Sunni Arab world.  It has also been lucky in that its local opponents have been weak.  It started out in Syria where it has been fighting the Assad regime.  There has been a full fledged civil war going on in Syria for more than three years now.  Sunnis in Syria see ISIS as fighting the Assad regime effectively on their behalf.  Syria is like Iraq in the sense that it is governed by a small minority.  Syria has large Sunni and Kurd populations both of whom have been oppressed.  ISIS has been able to leverage its position as "the Sunni savior" into effective control of a significant percentage of Syria.  At the moment the Assad regime is not in a position to do much about this.

Then early this year ISIS mounted an offensive into Iraq.  Theoretically Iraq has a large, well equipped, and well trained army.  But the problem is that The U.S. has insisted on a balanced force consisting of Shiites, Sunnis, Kurds, etc., all mixed together.  If it had worked this would be a good thing.  It is a key to the long term success of the Iraq that the U.S. envisions.  This Iraq would have its historic boundaries and would have all of its factions and minorities living together in peace and harmony.  But that is not the path Iraq seems to be on, no matter how fervently the U.S. wishes it were.  In the actual Iraq of today there is tremendous mistrust and a lot of jockeying for advantage.  In the Iraq of today the army is a hollow shell.

ISIS is not a hollow shell.  Their military forces are disciplined and effective.  ISIS was able to capture large swatches of Iraqi territory because they stuck to the Sunni areas where they received a friendly reception.  And they were opposed by a paper Iraqi army that dissolved under the slightest pressure.  The Kurds might have been able to effectively oppose ISIS.  But the al Maliki government  had starved Kurd military forces of supplies and equipment.  Why?  Because al Maliki did not want an effective and possibly hostile military force on his Northeastern flank.  So the Sunnis saw ISIS as liberators from the Shiite oppression of al Maliki.  The Shiites were defended by a hollow military force, the Iraqi Army.  And the Kurds had the will to fight but not the equipment and supplies.  It is not surprising that ISIS was able to rack up such impressive wins in the early days.  So what has happened since?

Lots.  The first thing that happened was that the Shiites had the pants scared off them.  Al Maliki had been involved in a nasty political fight for control of the official government of Iraq.  As "his guy" the Shiites were supporting him as were the Iranians.  So it looked like he would win in the end.  To the Shiites and the Iranians al Maliki was a good bet as a skilled inside fighter in the hard ball politics of Baghdad.  But the tactics he was using assumed that there was no serious outside threat.  With no serious outside threat it was ok to screw over the Sunnis and the Kurds and have an ineffective army.  But ISIS quickly convinced everyone in Baghdad that it was a serious outside threat.  That changed everything.

All of a sudden Baghdad was facilitating not impeding the flow of supplies and equipment to the Kurds.  And al Maliki stopped relying on the army.  Instead he turned to the all Shiite militias, many of whom had been trained and supported by the Iranians.  It was now more important to win military battles than political ones.  As a side effect of all this it looks like al Maliki is out.  With adequate supplies and equipment the fighting ability of the Kurds has miraculously increased immensely over night.  They were able (and willing) to organize the rescue of the Yazidis.  They have retaken territory from ISIS.  And it looks like they can retake even more.  The U.S. has continued to pitch in with intelligence and the odd airstrike.  And the airstrikes have continued to help.  But that is because there are boots on the ground to back them up.

Now let me back up even further.  There is a dirty little secret behind Arab extremism.  Most of it has been funded for decades by our "ally" Saudi Arabia.  The country is run by the House of Saud, the extended family of Ibn Saud, the first King of Saudi Arabia.  Back in the day the Saud family did a deal with the Wahhabi sect of Islam.  The family of Ibn Saud would be given a free hand to run the country.  In exchange the Wahhabis would be given control of the educational system and various "cultural components" of the government.  Specifically, they run the "religious police", who make sure that public behavior confirms to Wahhabi standards .  It will come as no surprise that the Wahhabis want to spread their ideas as broadly as possible.  They use their political power to do this.  They also encourage wealthy and/or powerful Saudis to donate money to advance the cause.  This has resulted in them being able to build and operate hundreds of Madrasas all over the world.  The Madrassas espouse Wahhabi ideas.  In a lot of the world the only way poor parents can get their children any education is to send them to one of these Wahhabi Madrassas.

The Wahhabis are very fundamentalist.  Perhaps their most well known adherent is Osama bin Laden.  Bin Laden's philosophy is well outside mainstream Islam but broadly compatible with Wahhabi beliefs.  So one of the big reasons that fundamentalism is widespread in the Arab world is the influence of the Wahhabi strain.  The Wahhabi strain has so much influence because it is so powerful within Saudi Arabia.  But we can't criticize Saudi Arabia because we depend on Saudi oil so heavily.  And it's not just Saudi Arabia.  This "buying off the religious people" tactic is practiced in a number of other oil rich Arab countries.  So we see Qatar and others doing the same kinds of things.  Qatar is another example of a tightly controlled government sitting on top of a country with a small population and a lot of oil.  You do what you have to do to stay in power.  So a dirty little secret of the ISIS magic formula is that it has been receiving a lot of support (usually quietly) from the Arab world.

So how do we deal with ISIS?  Step one is to stop listening to the "experts" on TV.  The rest of the plan comes straight out of the Obama play book, you know the one that the "experts" say doesn't exist.  Obama spent his first term repairing relationships with the rest of the world.  This is not always apparent because "take pot shots at the U.S." has been a popular pastime around the world for more than 50 years.  But the rest of the world knows they can deal with the Obama Administration.  The problem is that when things are going well it is usually more beneficial to give the U.S. grief.  But ISIS has changed the calculus.

ISIS represents a real threat.  So scoring a minor cheap shot off the U.S. is now counterproductive.  We have seen this most clearly with the Iraqis.  They have been giving the U.S. grief since Obama got into office.  If there is no down side then there is every reason to do it.  But now ISIS is knocking on the doors of Baghdad and threatening the flow of money, I mean oil, and threatening critical infrastructure like the Mosul dam.  So the Iraqi government is now cooperating with the U.S. and helping the Kurds out.  But we are also not getting any grief from Saudi Arabia or Qatar or any of the other Arab states.  ISIS is good because its Sunni.  But that was before it started slaughtering people in large numbers, enslaving people, and cutting people's heads off on the Internet.  Now it looks bad even to people like the Saudis

We have had an off and on relationship with Turkey.  Turkey would like to be seen as the political and cultural leader of the Arab world.  One way to do this is to be seen as the defender of all things Arab against western encroachment.  And the encroacher in chief is the U.S.  Getting too close to the U.S. is a bad for your image in the Arab world.  Turkey is no friend of the Assad regime.  But when it was deciding who to back against him it had lots of "rebel factions" to choose from.  I think ISIS was on its list of factions worth backing.  I don't think ISIS still is. And Turkey has a large restive Kurdish population.  Turkey has been shedding blood for decades trying to keep these Kurds in line.  So it seemed like a bad idea to do anything nice to the Kurds in Iraq.  But now it doesn't.  Why?  ISIS.  The enemy of my enemy is my friend.  All of a sudden the Kurds look down right friendly.

So since I have been bad mouthing what the "experts" have had to say, what do I think is the right thing?  First of all, most of the "right thing" is not stuff that generates "bang bang" footage.  It is the hard slog of diplomacy, of consensus building, of finding common ground, and building long term relationships.  That's what the Obama Administration has been doing and it is paying off.  What was not possible a few months ago is now possible.  And, if we keep our powder dry, even more may be possible in the future.

The key to our success is the bad behavior of ISIS.  The conventional wisdom as demonstrated by the  media is that we are smart and thoughtful and caring and the rest of the world is a bunch of dim witted cretins.  This is in spite of the mountain of evidence of stupidity, ignorance, and cretinous behavior on display within our borders on a daily basis.  We have figured out that the ISIS people are bad people.  The rest of the world is coming around to the same opinion.  Give them a little more time.  And in particular, give the people who live in ISIS controlled territory a little time to figure this out too.

Anbar used to ride high back in the day.  Saddam did a good job taking care of them.  He was a Sunni and they were Sunnis.  Then they ended up on the wrong side of the U.S. invasion.  We backed the Shiite majority.  So they reacted badly.  That put them in close contact with al-Queda in Iraq.  Initially this seemed like a good thing.  But over time the relationship soured and we had the Sunni Awakening.  Then the U.S. left Iraq and al Maliki started putting his boot on the neck of the Sunnis.  So ISIS initially looked pretty good.  But that will not last long.

These is a big problem here.  So far recent history has not presented the Anbar Sunnis with good options.  The key to cracking the ISIS nut is to give them a good non-ISIS option.  The same is true, by the way for the Syrian Sunnis.  So the long term goal is to get to a place where the Sunnis in the territory now controlled by ISIS are well governed.  By this I mean competently governed by local Sunnis.  It is not clear to me how to reach this objective.  But U.S. boots on the ground is nowhere on the path.  Here is a way forward that I can see happening.

Iraq gets partitioned into a Shiite state and a Kurdish state.  Syria gets partitioned into multiple states too.  I see no reason not to create a state consisting of eastern Syria and western Iraq, roughly the territory controlled by ISIS.  If you change the ISIS government for a moderate Sunni government you are home and done.  But what I am suggesting is not possible at the moment.  For various reasons it is critical at this time that the U.S. not support the partitioning of either Syria or Iraq.  So for the moment we need to continue to play the hand we have been dealt.

We need to do this or the Baghdad government will interfere with not facilitate the flow of aid to the Kurds.  Turkey is also informally cooperating with the Kurds but I don't think they are yet ready to go further.  Handled badly they could quickly go backward (a more hostile position toward the Kurds) as could the Iraqis.  Unfortunately this means that a bunch of Syrians and probably a goodly number of Iraqis are going to get killed, wounded, displaced, etc.  There are things that the U.S. could do that could theoretically mitigate the situation.  But those same things could also make things worse, much worse.  And it is important that everyone in the U.S. understand a fundamental truth.  There is a near unanimous belief in the area that we have interfered too much in the region.

We have a large debit balance in Iraq according to Iraqis.  We tend to look at how many Americans died and how much money we spent.  Iraqis tend to look at the fact that we went where we were not invited and then made a big mess.  Iraq was laid waste.  Delicate balances were upset.  Many thousands of Iraqis were killed.  Millions were displaced.  Whether we like it or not al Maliki was seen as "our man in Baghdad".  So we get the blame, whether we deserve it or not, for a lot of his actions.  We have built up some credit here and there in Iraq and elsewhere in the Arab world.  But we are still deep in the hole.  The best way to be seen as digging the hole deeper is to be seen as either going it alone or going against the wishes of the locals.

We must work through the locals.  We must be seen as assisting the locals in whatever they want to do.  We must be seen as NOT dictating.  Dictating is heavy handed.  The Bush administration gave the region and the world far more heavy handed than it wanted to deal with.  Obama has mitigated some of the damage.  But we are not even back to dead even yet in the region or the world.

The trick is to let events come to us, not the other way around.  Bad behavior by ISIS creates opportunities for us.  If we are seen to be continuing to work through Baghdad and the Kurds and the Turks then many things become possible.  Then we work behind the scenes through the locals.  That means we are limited to actions that the locals are ok with.  Ultimately, that disciplines us in a good way.  It's a frustrating and slow way to work.  It will cause talking heads to continue to heap scorn on President Obama in spite of the successes he achieves.  But it will eventually yield the best results.

Do you think ISIS will continue to misbehave?  So do I.  That will open up more possibilities if we are just patient.  Currently Baghdad is opposed to a partition of Iraq.  Let's see how things look in a year or so.  Currently Baghdad and Turkey and lots of other Arab countries are opposed to an independent Kurdish state.  Let's see how things look in a year or so.  Currently ISIS is in complete control of its territories.  Let's see how things look in a year or so.  If the locals decide that a partition of Iraq is a good idea then we can facilitate that.  But we can't impose either a united Iraq or a partitioned one from the outside.  Continuing to try will result in frustration in the short run and failure in the long run.

I don't know what the state of play is with respect to Saudi Arabia, Qatar, etc. and their funding of extremists.  I mistrust what little coverage of this that I see in the media.  Let's see what the state of play is in a year or so.  Frankly, the U.S. has never pushed the issue of the funding of fundamentalists with any of the Arab states that do it.  That's because oil concerns override everything else.  But what if these states decided on their own that it was time to stop?  Again, good things that are now not possible become possible.  And what if an indigenous Sunni reaction materializes within the ISIS region?  It has happened before with the Sunni Awakening.  Then we have something to work with.  The difference is between trying to bludgeon someone into doing what you want and trying to gently encourage them.  People don't like to be bludgeoned.  They tend to push back even if they know what you are trying to get them to do is a good thing.

The Arab world has been governed very badly for a very long time.  The Arab Spring is a reflection of a realization on the part of the Arab street that this is so.  Getting from bad government to good government via revolution is a very hard thing to pull off.  We tend to focus on the American Revolution where things went well.  But what happens more often looks more like the French Revolution.  One of many bad side effects of that experiment is the Napoleonic Wars.  France did come out the other end eventually.  But it was a very long and painful process for France and for all of Europe.  Both Libya and Egypt seem to share more similarities to the French example than the American one.  And Iraq may follow the French model too.

At this point someone always wants to bring up Ruanda.  There was a large genocide there in the '90s.  The feeling now is that if the west had intervened at the proper moment then the genocide could have been avoided.  And in the case of Ruanda the feeling is probably correct.  But circumstances are always different.  In Ruanda the theory is that all that would have been necessary was for the west to make a lot of noise.  Little or no actual action would have been required.

The U.S. had good intentions in Iraq and it invested a lot of time, money, and people.  It failed anyhow.  We made Iraq a mess.  The one thing people agree on with respect to Syria is that a lot of time, money, and people would be required to make a big difference.  The ISIS territory is surrounded by Arab states.  There is no way to get to it without going through one or more of them.  Any large unilateral effort by the U.S. is doomed to failure.  Not every humanitarian crisis is avoidable.

Since we are doomed to failure without local cooperation why not put local cooperation on the front burner and immediately jettison any idea that sounds like "go it alone"?  I have found it a valuable rule of thumb to ask how much cooperation is actually on offer in these situations?  If the answer is "lots", as it was in Iraq I, then prospects are good.  If the answer is "not much", as it was in Iraq II, then things are very likely to go very wrong.  A few months ago there was very little real cooperation on offer in the region.  There is now a lot more.  By treading carefully and by doing less rather than more the amount of real cooperation on offer looks to keep increasing.  This is definitely a case where being late is much better than being early.