Positive Identification

"Who are you?"
"Jane Doe."
"Prove it!"

Some variation of the above dialog is now a common part of our lives.  It is frequently boiled down to "Show me your picture ID."  The picture ID contains a name, a picture, and typically other information.  The name provides the answer to the question.  The rest of the information on the ID and the fact that you possess it provides the proof.  The connection between you and the ID is provided by the picture.  Presumably you and your picture can be compared to see if there is a match.

There are variations.  It is becoming more and more common for your smartphone to stand in for your picture ID.  And the degree to which the "proof" actually validates your identification varies.  Bartenders just want to know if you are old enough to drink legally.  The TSA wans to be really sure you are not a terrorist.  And then there is the sad situation with which the title of this post is most commonly associated.  Someone may need to confirm the identity of a deceased person.

With the background established let's look at the process of positive identification as it was, as it is, and as it will soon be.  The times they are a changing.  Let's start with the "was" part and for that I want to go back a thousand years.

A thousand years ago almost everyone lived on a small farm or in a small village.  Almost everyone a farmed, fished, or was otherwise engaged in the process of growing and harvesting food.  And at the time almost everyone was illiterate.  Paper hadn't been invented yet.  The alternatives that existed at the time (i.e. parchment) were all extremely expensive and only available in tiny quantities.  So in that environment how were people identified?

Most people spent their entire lives within a few miles of where they were born.  Everyone knew everyone else in the neighborhood by sight.  You saw people at home or on market days or at feast day events.  And you saw them for their entire lifetime.  At some point a woman would be pregnant.  Then she would show up with a small child.  The child would grow up, become a parent and die.  And the community observed all this.  Identification was not absolute but it was good enough for the situation.  You knew who farmed what piece of land and who their children were.

And frankly positive identification was not that important.  People were poor so they had few possessions and even fewer valuable ones.  Land ownership was mostly governed by the "possession is nine points of the law" rule.  There were no accurate surveys and all the land was probably technically owned by the local feudal lord anyhow.

Oh, there were foreigners.  Someone from outside would occasionally wander by but this did not happen often.  If the wanderer was a trader how much did it matter who they were?  They showed up, traded, and moved on.  The trade goods were important.  The identity of the trader was not.  The other group who would show up occasionally were members of the power elite.  It might be a soldier or a priest.  If a particular soldier was the top dog he became the local feudal lord.  Other soldiers either worked for him or there was a power struggle.  Eventually someone came out on top and the others ended up dead, part of the lord's operation, or they moved on.  The feudal lord was in a position to assert his authority by means of his ability to kill or maim you.

So the locals tended to take him at his word as to who he and what he was.  The niceties of the law and who's authority was more legitimate tended to be less important than who won the power struggle.  The other source of authority and power were the religious authorities like priests.  If there was a power struggle between religious factions the rules of engagement were different (less blood more politicking) but who stayed and who was pushed out mostly depended on who was supported by the local feudal lord.  And again, the peasantry tended to take whoever won at their word.  So in this period positive identification had little real practical meaning.

Eventually paper got invented and the technology for making it cheaply spread broadly and it became practical to keep paper records.  This ushered in the era when marriages, births, and deaths, started to become routinely recorded.  For a long time the process was hap hazard.  A record might be maintained at the local church or in a family bible.  How reliable was this information?  One assumed that it was fairly reliable.  But this assumption rested to a great extent on past practice.

Usually people in the community were around to testify to the accuracy of the information, at least until enough time had passed that all the eye witnesses had died.  After that inertia set in.  Records that had been accepted in the past continued to be accepted.  Beyond that, old records came to be seen as accurate records, mostly because they were old.  And it was certainly possible for a record to be fudged.  A marriage could retroactively be added to a church register or a family bible.  And the same process could be used to erase or alter entries.  People went with these records as much because they were the only practical option as for any other reason.

Not that long ago governments started taking over responsibility.  They started issuing birth certificates, wedding licenses, and certificates of death.  And more people were born in a hospital with a physician in attendance.  But in a certain sense the foundation the process rested on had not changed.  Someone filled out a form.  The information was only as dependable its source and its source was some person.  The person might be the mother or the doctor or a hospital employee.  And, in the case of the doctor or hospital employee, they might be relying on some stranger for the information they were entering.  I suspect that most of the time little effort was made to corroborate it.

The piece of paper has now been replaced by a computer screen and the data no longer resides on a piece of paper in a file cabinet.  It now resides in a computer file somewhere.  And this highlights a fundamental problem.  It's just data.  And more problematic than that is this.  How do we know that a particular birth certificate is actually the birth certificate of a particular person?  The surprising answer is that we don't.  But that can, and I expect that it will, change in the near future.

We have all been exposed to this sort of thing due to the "birther" controversy.  A lot of people but most notably Donald Trump have spent a lot of time and gotten a lot of media coverage contending that President Obama was not born in Hawaii in 1961.  A lot of their argument is nonsense.  There is absolutely no doubt that a birth certificate was issued at the time and place the President contends it was.

An argument could be made that he is not the child that belongs to that birth certificate, that there were, in fact, two children.  This argument is logically consistent but it is not the argument that birthers make.  And there is a large body of evidence that there is only one child and he is that child.  In fact the connection between this birth certificate and the President is much stronger than the connection between Donald J. Trump and any birth certificate.  So the birther argument, such as it is, is about the wrong thing.  A fundamental question exists.  How do you definitively connect any person with any birth certificate?  And the answer is that in almost all cases you can't.

In some places at some times a footprint (like a fingerprint but of the bottom of a foot instead of the fingertips) of the child was routinely put on the back of birth certificates.  With such a birth certificate you can take a matching print of the foot of the individual in question and do a "fingerprint analysis" to see if it matches.  If it does then you can definitively match a specific birth certificate to a specific individual.

But I have never heard of this comparison being attempted.  As far as I can tell the "footprint on the birth certificate" procedure was never common and, in the cases where it was done, I know of no instances where a match was attempted later.  It probably happened but it was never common enough to feature in crime fiction, for instance.  It is easy to imagine Erle Stanley Gardner plugging it into a Perry Mason novel but he never did.  The fact that it was never a common crime fiction motif is evidence that it was never a common practice.

I have both a passport (expired) and an "enhanced" driver's license.  Both of these require a positive identification.  Having been there and done that I know the drill.  I show up, fill out some paperwork, provide a picture (or get one taken), and provide "positive identification".  What's positive identification?  Why a birth certificate, of course.  So I hand over a piece of paper for examination by a bureaucrat.  But what's the piece of paper?  In my case I have the original actual birth certificate that was issued at the time of my birth and it's a pretty ordinary looking piece of paper.  That's how it was done in the era that preceded the computerization of everything.

But I actually have two "birth certificates".  One of them is the aforementioned piece of paper.  The other is a "certified copy" of the piece of paper.  It is something called a Photostat.  A Photostat, as you can guess, is just a photograph, well actually a print of a photographic negative.  The only thing special about it is that it is embossed with an official stamp and an "I attest that this is an authentic copy of . . ." statement followed by the signature of some obscure bureaucrat.

Let's say I wanted someone to impersonate me.  I could keep my original birth certificate and give them the Photostatic version.  They could use that as the basis of a scheme to identify themselves as me.  So there could be two official me's running around.  And, in fact, a minor variation of this used to be commonly done.

People who wanted to change their identity would search newspaper death notices for someone who was born about the same time they were but who had died young.  They would then write the proper authority asking for a Photostatic copy of the birth certificate for this person.  At the time this was a routine bureaucratic procedure that did not require any kind of special documentation.  When it arrived they would then use the Photostatic birth certificate as the base on which to build up an entire false identity for themselves.  If they picked their dead person properly their chance of being caught out was infinitesimally small.

Spies, crooks, people on the run for political reasons, etc. did this routinely in the '60s.  You could even find "how to" manuals if you knew the right people.  One thing that helped then was that most people did not get a Social Security card and number until they entered the job market in their middle to late teens.  If you picked someone who had died at ten, say, your chances of fooling the Social Security Administration into issuing a card were very good.  It is now much harder to pull this kind of thing off because we are all now surrounded by a much larger more complicated web of interconnection than we used to be.

Children now get issued a Social Security number at birth, for instance.  And as big data spreads its tentacles it becomes harder and harder to pull something like this off without setting an alarm off somewhere.  The federal witness protection people can still do it.  But they can change Social Security and other government records.  But none of this changes the fact that there really is no completely reliable way to positively connect a specific birth certificate to a specific person.  But I believe that is going to change in the near future.

There now exists something called CODIS, the Combined DNA Index System.  This is the database that is used to do DNA matches in crime scene and other law enforcement (i.e. missing persons) situations.  The database contains over 12 million entries and continues to grow rapidly.  There is a considerable amount of duplication so it doesn't represent that many distinct individuals but the number of distinct individuals is somewhere in the millions.  Each entry contains enough information to identify a single specific individual with very high degree of confidence.  Does that mean entries contain complete DNA sequences?  Far from it.  Instead each entry contains 14 numbers.  Thirteen of the numbers are based on something called a STR, a Single Tandem Repeat.  The specifics are complicated but the idea is simple.

An STR "locus" is a very short piece of DNA that varies wildly from person to person.  There are a bunch of variations possible for each STR locus.  The database contains the specific variation number in the DNA of the entry for each of the 13 STR loci.  The 14th number is based on a person's Amblogen gene.  It has been included because the version of Amblogen gene that an individual has tells us whether that person is a male or a female.

Only a few percent of the population has a specific version of a specific STR locus.  So different individuals are likely to have a different variation of the first STR locus.  But they could just by luck have the same variation.  But do they also have the same variant in the case of the second locus?  Here too it is very unlikely that two different people have the same variant but it is possible.  And so it goes.  Scientists have done the math and the likelihood that two different people who are not identical twins would have the same variant of all thirteen STR loci is a really tiny number.  It varies from case to case but it is unlikely that two non-twin individuals on Earth have the same variant of all thirteen STR loci.  And just to decrease the chances even more there is a move afoot to add several more STR loci to the standard list.

It turns out that the amount of DNA in all fourteen loci used in this process is a tiny fraction of your whole genome.  It's way, way, way less than 1%.  But it is enough to get the job done, namely deciding if two DNA samples come from the same person or not.  And the basic technology for this was developed more than a decade ago.  In the mean time anything having to do with DNA has gotten a lot cheaper.

The original project to sequence the entire DNA of a single individual cost more than 3 billion dollars and took about a decade.  Now the complete DNA of a single individual can be done for about 10 thousand dollars and it's getting cheaper every year.  Scientists think the cost will drop to below a thousand dollars within the next few years.  And that's what it costs to sequence everything.  The cost to sequence enough DNA to tell one person from another costs way less than that and that cost is also dropping like a rock.  And of equal importance the size of the gadget that does the CODIS sequencing is also getting smaller and smaller.  And that opens up a lot of possibilities.

We as a society have been fighting over privacy for a long time now.  Before the Revolutionary  War colonists decided they didn't like British soldiers searching peoples homes any time they wanted to.  They complained about it in The Declaration of Independence.  After the War the US adopted the Fourth Amendment outlawing "illegal search and seizure".

When I was younger we were fighting the Cold War.  The USSR was an "authoritarian dictatorship".  The Nazis before them were also an authoritarian dictatorship.  Both regimes were famous for requiring everybody to carry "papers" that had to be produced any time any place any time any official wanted to examine them.  So, since we were the good guys, we were all in favor of the opposite.  Our citizens were able to move about freely and were not be under any obligation to produce their papers.  It was a point of differentiation between us and them.  "Only authoritarian dictatorships require law abiding people to always carry identification documents as they go about their ordinary business."

Well, times have changed.  The USSR is no more so apparently we no longer need to differentiate the behavior of our government from that of authoritarian dictatorships.  It seems that we are now all supposed to be afraid of terrorists in our midst.  And that means anyone who is suspicious (not a well dressed white person) had better have their papers on them at all times.  And besides terrorists there is the ever present danger of rapist Mexicans or whoever else fits the "looks suspicious" profile.  I am going to ignore the issue of whether this change is a good thing or a bad thing.  Instead I am going to focus on the technicalities of how to positively identify people.

As I have discussed extensively above, the birth certificate is the foundation of identity for US born individuals.  There is an elaborate system in the US for dealing with the foreign born that I am not going to get into.  I will just note that in many cases it often ends up coming back to a birth certificate for these people too and move on.  And, as I have also extensively elaborated on above, there is no way currently to definitively tie a specific individual to a specific birth certificate.  And by now I think I have telegraphed where I am going pretty clearly.  The thing that could tie the two together is CODIS style DNA information.

There is no technological impediment to doing this now.  A sample sufficient to the task is easily obtained from a newborn.  Blood works and only a drop is necessary.  And the equipment needed to take the necessary measurements is relatively inexpensive and the process is relatively quick.  So it is completely possible to CODIS characterize every newborn at birth.  (As a side note it is also easy in most cases to CODIS characterize the mother and, if he is handy, the father at the same time.)  And the amount of data is modest so it could easily be added to the birth certificate computer record.  Once this is routinely done and some time has passed it becomes a simple process to prove that a specific individual is the one connected to a specific birth certificate.  You just draw a drop of blood, run it through the CODIS process and see if the results match the information in the birth certificate record.  None of this is beyond our current technical capability.

But it is currently beyond our political capability.  People do not want to be in the CODIS database.  Part of this is due to the association between the CODIS database and criminality.  But a lot of people see it as an invasion of privacy they are unwilling to put up with.  They can be convinced to change their mind if there is great need, say a loved one is missing.  But currently every state has restrictive policies that limit who goes into the CODIS database.  Not even all criminals or suspects go in now.  The details vary from state to state.  Some have restrictive policies and CODIS only a relatively small number of people.  Others apply a broad brush and CODIS many more.  But all states prohibit adding people without cause.

And the CODIS database is not the only DNA database in existence.  People sign up with 23andme or other similar companies that do DNA analysis.  The company tells them, for instance, where their ancestors are from.  Various groups also collect DNA information for a number of different scientific reasons.  But both the commercial and the scientific operations are careful to not sequence the DNA loci that CODIS uses.  They just don't want to get tangled up in criminal investigations.  And the people whose DNA ends up in these other databases like it that way.

But let me emphasize that this is a decision that is made for non-technical reasons.  Companies like 23andme try to retain the original sample so that it can be reanalyzed as technology advances.  So they could easily reanalyze the samples they still have and sequence the CODIS loci.  The sequencing they already do is much more extensive than what the CODIS process requires.  And if they did this their database could be used for CODIS-compatible searches.  The number of people whose DNA could be CODIS matched would immediately jump substantially.  But this is not really necessary.  There is already a strong trend in place to keep expanding the CODIS pool.  It is partly a result of technological considerations.  It keeps getting quicker, cheaper, and easier to CODIS samples.  And the people that run CODIS type databases keep coming up with more and more reasons to include more and more people in their collection programs.

I would think that intelligence agencies like the CIA would want to CODIS their employees and contractors.  And how about soldiers?  And how about law enforcement people.  And, on the other side, how about foreigners entering our country.  And how about people busted for minor offenses like speeding tickets or people involved in divorces or people filing for a business license or people involved in food preparation or, or, or.  As the ease with which the process can be performed and the cost comes down the strength of the argument necessary to justify including an additional group gets less and less.  And as this trend continues at some point you will have twenty or thirty percent of the entire population in the database.  At that point you might as well just put everyone in.

Consider that many crimes now go unsolved.  There is DNA evidence available in many of these cases but it doesn't match any entries in the current CODIS database.  If we had CODIS coverage of the entire population then it would go some way toward increasing the percentage of crimes that do get solved.  This higher solution rate should lower the overall crime rate, right?  And isn't lowering the crime rate a laudable goal?  That is only the most obvious potential benefit to CODISing everybody.  Other potential benefits are easy to come up with.  Instead of listing them let me extrapolate a little ways into the future.

When I was younger pretty much all small transactions (i.e. buying a cup of coffee) were done with cash.  Then people started using debit cards instead.  There are now a lot of people who carry only a small amount of cash around.  And as I write this we are transitioning to an even newer method, paying with our smartphones.  Today it is rarely used (except at Starbucks).  But that is because there are some kinks that need to be worked out.  Not all smartphones work at all stores all the time.  That's mostly because we have dueling incompatible payment systems fighting it out.  And for business reasons each system makes sure that it is incompatible with any of the other systems.  At some point that competition between systems will be made to stop.  Then people will be able to use one application on whatever phone they like to buy stuff from whoever they want to.  But that puts the identification issue front and center.

The simplest thing from a user standpoint is to always leave your phone unlocked.  And far too many people do this because dealing with the security system is bothersome.  But Apple came up with a trick.  You put your thumb in the right place and the phone can validate your thumbprint.  This can be done almost instantaneously.  And this approach is now being copied by the other smartphone makers.  I expect it to be universal within a few years.  But I suspect that the thumbprint scheme is not really that secure.  The phone only sees part of your thumb and in poor conditions.  The vendor (e.g. Apple) does not want a bunch of false negatives (you put your thumb on your phone but it doesn't okay you) so I suspect that the phone calls anything that is even vaguely close a match.

But let's fast forward a few years.  Currently the easiest way to do a CODIS analysis is with a drop of blood.  But with a lot of effort even very tiny amounts of DNA can sometimes be used.  In ideal circumstances the tiny amount of DNA that ends up in some fingerprints is enough.  And it turns out that there are lots of cells on the surface of your skin that contain your DNA.  (That's where the fingerprint DNA comes from.)  These cells can be collected and processed without having to poke a hole in you, a process that is not very painful but "not very painful" is not the same as "not even noticeable".  And it is easy to imagine harvesting a few cells from the surface of your finger in a way that is not even noticeable so let's imagine it.

Next imagine the CODIS analysis device being small enough and cheap enough to be incorporated into a smartphone.  And, while we are at it, assume it can produce an accurate result in less than a second. Now we have everything we need to build a system right into our smartphones that is fully capable of positively validating that you are you.  And it is quick enough so that it can be used routinely, perhaps a hundred or more times per day.  That would definitely solve the positive identification issue for smartphone transactions.

I think that for better or worse this is the direction we are heading.  I would like to say that it is not inevitable but I am concerned that the forces that are pushing in this direction are powerful enough to overwhelm any opposition I can currently foresee.  I think most people will be of the opinion that it is no big deal.  In the fight between Apple and the FBI over unlocking that iPhone (see http://sigma5.blogspot.com/2016/02/digital-privacy.html for more on this subject) that was the opinion of a large segment of the general public when they were surveyed on the subject.

They put it another way:  "I've got nothing to hide so what's the problem?"  That situation did not seem to directly affect them.  They did not foresee the FBI or anyone else wanting to unlock their phone so it didn't seem personally important either way.  In the case of what I am now taking about the direct connection is much more obvious.  But there are also immediate benefits.  "I can use my smartphone to pay for my coffee without having to worry about someone maxing out my credit cards if my phone gets stolen."  (As a side note if smartphones used this system they would be useless to thieves and thieves would stop stealing them.)

Our privacy is continuously under assault.   Technological advance keeps making it easier to invade our privacy and harder to protect against an invasion.  If everyone ends up in a CODIS-type database and that database is routinely used to confirm our identification and if a truly positive identification is the norm then pretty much every nook and cranny of our lives will be stored away in one or more computer databases.  It looks like this eliminates any technical barrier to the complete invasion of our privacy.

I'm sure at least some will continue to say "I've got nothing to hide."  But that's not really true.  You may think you have little or nothing to hide.  But all of us have opinions and all of us lead our lives in certain ways.  Bear in mind that whatever opinions you hold there are a large number of people who think you are wrong.  And no matter how boring you think your lifestyle is there are lots of people who strongly disapprove of it.

Are you a girl who likes to wear pants?  Are you a guy who likes to shave?  There are people who are seriously unhappy with you.  What religion to you follow?  It doesn't matter.  There are a lot of people who hate that religion, whichever one it is.  Do you like city living or do you prefer the wide open spaces?  Either way, there are people who are seriously unhappy with you.  Those are all choices many people would find boring and unimportant.  How about more controversial ones?

Do you drink?  Have you ever had sex outside of marriage?  Have you tried non-missionary sex?  Have you smoked pot?  How about other drugs?  Even once?  Have you ever broken a traffic law, driven drunk, or maybe after you have had only one or two?  Have you ever skinny dipped or streaked or done anything else "young and stupid"?  Have you ever stolen something, even accidently?

The point is we have all done some embarrassing things, maybe even a lot of embarrassing things.  And we have all done things some would disapprove of to the point that they would delight in harassing us about them.  So we all have things to hide.  Pretty much all of us have things we would prefer our parents, or our children, or our friends, or our coworkers, or the authorities, or our enemies, or random obnoxious people we don't know, don't know about.  In other words, we all value out privacy.

In the past there have been practical or technological barriers we could hide behind.  The tatters that remain of the old barriers are quickly being shredded.  I have addressed the general issue of privacy before (see http://sigma5.blogspot.com/2013/12/privacy.html).  I devoted roughly the last third of that post to what I thought should be done.  I wrote that post over two years ago.  The current topic only adds to the pressure that is moving us toward a world where there is no privacy.  I recommend that post for my overall thinking on what should be done.  Meanwhile there is a small piece of good news on the privacy front.

I linked to my blog post on the fight between the FBI and Apple above.  At the time I wrote it no one knew how it would come out.  But that specific situation has since been resolved.  The FBI found a way to crack the phone that did not require the extraordinary cooperation that Apple was objecting to.  That sounds like bad news but it's not.  The phone that was cracked is an older model.  Apple has upped its game with newer models.  Whatever methods were used are unlikely to work (or at least will be much harder to pull off) on newer models.  And in spite of various polls that were done at the time it turns out that there is a market for secure phones.  So Apple has promised to keep adding features to make each new generation of phones much harder to crack than the old generation.  And remember the phone the FBI was only able to crack after a great deal of difficulty is now a couple of generations old.

And various other technology companies are now jumping onto the "increased security" bandwagon.  They are encrypting more and encrypting to a higher level of security.  They are also changing how their products operate so that they no longer have a backdoor that lets them read unencrypted customer data.  This means that if they are subpoenaed they can respond "sorry -- we can't read it either".  And a side effect of this is that they can't sell or analyze detailed customer activity like they used to be able to do.

They can still do a metadata analysis.  For instance they can figure out who you are interacting with.  They can tell how often you are connecting up and how long you are staying connected.  But they can't tell what you are doing while you are connected.  This means that the data they can share with someone else, the government or another company, is much more limited than in the past.  And that means it is much less valuable.  And that means they will do less sharing in the future.  And that is a modest step in the direction of more privacy.  It is a small but very welcome development.

50 Years of Science - part 6

This is the sixth in a series.  The first one can be found at  http://sigma5.blogspot.com/2012/07/50-years-of-science-part-1.html. Part 2 can be found in the August 2012 section of this blog.  Parts 3 and 4 can be found in the September 2012 section. Part 5 can be found in the March 2016 section.  I take the Isaac Asimov book "The Intelligent Man's Guide to the Physical Sciences" as my baseline for the state of science as it was when he wrote the book (1959 - 1960).  More than 50 years have now passed but I am going to stick with the original title anyhow even though it is now slightly inaccurate.  In these posts I am reviewing what he reported and examining what has changed since.  For this post I am starting with the chapter Asimov titled "The Birth of the Solar System" and then moving to "Of Shape and Size".  Both chapters are in his "The Earth" section.

The first chapter under discussion doesn't even mention the Earth.  It reviews various theories about the formation of the Solar System.  If you want to know what science looks like when Science has only a vague idea of what it is talking about this is a good chapter.  This chapter was written at the dawn of the space age.  I have talked about the best device for studying the heavens at that time, the 200" Hale telescope, elsewhere.  Frankly it was not up to the task of studying the Solar System in the detail necessary to understand it the way we do now.  Scientists of that time knew the size and orbital parameters of all the planets.  Asimov lists some very important observations that scientists had picked up on by that time.

Nearly all the planets had circular orbits.  (Pluto, the only exception, had not yet been demoted from planet-hood back then.)  All the planets orbited in a counterclockwise direction (when looking down from a great height above the Earth's North Pole).  Nearly all the planets and nearly all the moons known at the time rotated in a counterclockwise direction around axes that were roughly vertical.  (There were a few exceptions but they could be explained away as "exceptions that proved the rule").  And with each planet (again excepting Pluto), the ratio between the size of adjacent orbits fell in or near pleasant ratios.

There seemed to be a system to the Solar System.  But scientists were pretty much stumped as to what that system was.  The best theory at that time was one by Weizsacker.  His 1944 theory had serious problems but it was the best anyone had come up with.  So what, in the most general sense, was the problem?

There were two problems.  The obvious one is the one I have already alluded to.  They didn't have much data.  The Hale telescope was better than nothing but not that good at making the necessary precision measurements.  There were a few satellites in orbit but none of them had a telescope or other good instruments for studying the Solar System.  There may have been probes launched toward Venus or Mars (I didn't check) but, if so, they were very primitive fly-by missions.  And pretty much nothing was known about the gas giant planets.  The first great exploration missions to them, Voyager I and II, would not even launch until 1977.  The same was true for missions to the rocky inner planets (Mercury, Venus, and Mars).  And the greatest instrument of them all, the Hubble Space Telescope, did not launch until 1990 and it took a couple of years more to fix it.  So scientists lacked data.

They also lacked analytical tools.  There were some computers around in 1960 but they were small and slow by modern measures, and also few and far between.  I personally own three desktop computers.  Any one of them was more powerful in terms of speed, RAM, and disk space than all the computers in existence in 1960.  This meant that scientists were stuck with literally not much more than the back of an envelope when it came to thoroughly investigating a theory or trying to assess its ramifications.

A good thing to use as an illustration is the orbital spacing I mentioned above.  The spacing of the orbits of the planets was known to a medium degree of accuracy.  But the idea of "resonance" was not well understood.  Imagine two bodies orbiting the same larger body.  And imagine their orbits are such that one takes exactly twice as long as the other.  This means that over the course of one of the slower orbits each and every combination of relative position will happen exactly twice because the faster body will have orbited twice around and both bodies will end up exactly where they started with respect to each other.  Now imagine that there is a certain configuration where the bodies tug on each other, pulling each other in one direction.  It turns out because of the complete symmetry of the situation that there is another configuration where they pull in exactly the opposite direction.  So over the course of one slow orbit everything exactly balances out.  This is called a 2:1 resonance.

Now assume the resonance is slightly more than or less than 2:1.  Then a net pull can develop over the course of a complete slow orbit that slows one planet down a little or speeds it up a little.  This means that over time the planets will be pulled a little closer together or a little farther apart.  In other words, in our 2:1 resonance case the orbits are stable (they don't change at all over time) but in the other case they evolve.  Now there are other resonances like 3:2 or 4:3 or whatever.  With lots of cheap computer power astronomers are now able run sophisticated long running simulations to discover exactly how things would evolve.  The current state of the art now permits very complicated situations to be thoroughly analyzed and understood.

And with this ability astronomers found that there were only a few stable resonances.  The rest of the time the planets (or moons) get pulled around, often in complicated ways that could not have been predicted by looking at the equations and doing some simple analysis.  The simulations showed that they kept getting pulled around until they hit a stable resonance point, a point that may only have been arrived at after the simulation had covered millions of simulated years.  And guess what?  The current orbits of the planets are predicted by this resonance point analysis.  It was literally impossible to do this kind of analysis before abundant computer power was available cheaply.

The other problem is data.  As we sent space missions like Voyager out we were able to gather tons of data that was much more accurate and complete than that available in 1960.  This was the information needed to do the resonance point analysis with enough accuracy to give meaningful results.  It was literally impossible to make the kinds of detailed calculations necessary unless the parameters put into the simulation were known to a very high degree of accuracy.  Those highly accurate values were not known until we had sent spacecraft out exploring.

And this led directly to one of the things that astronomers got wrong at the time.  That was the question of the origin of the asteroid belt.  The asteroid belt (there are actually several but I am going to concentrate on the main one) consists of a bunch of sub-planet-sized rocks.  The leading theory of the time was that something had torn up a small planet.  We now know that the asteroid belt is a side effect of these resonances.  But the reason we now know this is because we have a lot more data and the data is of much higher quality.  We can also simulate the creation or destruction of a single larger body.  The simulations can't be made to produce the outcome we now see.  But a simulation of a bunch of rocks shows them drifting into the area now occupied by the asteroid belt then getting stuck there.

It wouldn't work if it was just one rock of whatever size.  A single rock would be pulled either toward Jupiter or toward Saturn.  But a flock of rocks can be stable over long periods of time.  At any one time some rocks are pulled in and others are pulled out.  But on average and over time, they just stay within the band that is the belt.  The average can maintain a behavior (stability) that no individual component is capable of.  Individually their orbits are all slightly unstable but this leads to stability at the group level. 

And now we have the instruments to study the individual asteroids in the asteroid belt in considerable detail.  NASA has recently inserted a space probe directly into the middle of things.  The Dawn mission put a spacecraft into orbit around Vesta, a large asteroid.  After studying Vesta for several months the probe was moved to Ceres, the largest asteroid.  Dawn has returned a massive amount of data about the asteroid belt to supplement what earlier probes discovered.

A couple of decades after 1960 scientists thought they had a good handle on the formation of the Solar System.  The idea was that the Sun condensed out of a cloud of gas.  This happened precisely 4.567 billion years ago.  (They have good reason to believe they know the Sun's age that accurately.)  They also think the rest of the Solar System formed a very quickly and a very short time later.  It only took a hundred million years, give or take.  They are very certain that it was very quick but exactly how quick is not nailed down very well.

And they had a theory which sounded very good about why the various planets with their various compositions ended up where they were and with the composition they did.  The theory was that the planets formed in roughly the locations you now see them.  The heat of the Sun's radiation was enough to blow the gas out of the inner solar system and into the outer solar system.  So you had rocky planets (Mercury, Venus, Earth, Mars) in the inner solar system and gas giants (Jupiter, Saturn, Neptune, Uranus) in the outer solar system.  Pluto was assumed to be an asteroid-like thing that got knocked around until it ended up where it ended up.  This theory sounded reasonable to everybody and seemed to work very well.

Then it became possible to discover exo-planets, planets orbiting some star other than our Sun.  The Kepler spacecraft has found literally thousands of them.  And the solar systems around these other suns don't look at all like our solar system does.  There are gas giants in the inner solar system all over the place.  Lots of gas giants have been found with orbits that are smaller even than Mercury's.  What astronomers now know for sure is that don't know.

The common theory for the moment is a hybrid one.  Planets formed in their traditional locations.  Rocky planets formed close in (they are hard to see if they are orbiting another start so it is no surprise that very few have been discovered).  Gas giants formed further out.  Then the gas giants migrated (see the discussion of resonance above) into the inner solar system of these other stars. In this theory the fate of the rocky inner planets is unknown.  But frankly this is a theory like the ones discussed in this chapter of Asimov's book.  It has problems but it is the best scientists currently have.  This means scientists expect the theory to undergo drastic modification or even be discarded completely for a quite different one.

"Of Shape and Size" starts with a discussion of the shape of the Earth.  It has been known to be roughly spherical for several hundred years now.  But some noticed evidence supporting the idea of a spherical shape much further back.  But for a long time their evidence did not carry the day.  By Newton's time (400 years ago) it was generally accepted that the Earth was spherical in shape.  But Newton calculated that gravitational effects should distort it into an oblate spheroid.  The French in the 1800s tried and eventually succeeded in confirming Newton's idea.  The best number for exactly how far out of round the Earth was in 1960 was 26.7 miles.  That is not far off the current number.

We now have much more accurate ways of measuring distance.  So we can very accurately measure the distance from a fixed point on the Earth to a satellite.  A bunch of these measurements yields a very accurate description of the exact shape of the Earth.  It is an oblate spheroid with a number of lumps and bumps on it.  The actual shape, even after you smooth out mountains and oceans, is very complicated and I am not going to go into it.  And, of course, we have turned the whole "satellite distance" thing around to create the GPS system.  GPS satellites need, among other things, a mathematical model of the shape of the Earth.  They use a moderately sophisticated one that works well enough to keep our navigation systems on track almost all of the time.

One of the things the French effort brought out, Asimov tells us, is the fact that at the time there was no agreed upon standard of length.  Everybody knew approximately how long a yard was but no one knew precisely how long it was.  This led to the creation of the "Meter" (French spelling:  Metre).  It was the distance between two very precisely marked lines on a specific piece of metal.  Eventually the "Metric standard" was adopted around the world.  Now even the Yard is defined in terms of the Meter.  An "Inch" is one 39.34th of a Meter.  A "Yard" is 36 inches.  It's clumsy but it works.

And this "two marks on a piece of metal" definition of the Meter worked well for more than a hundred years.  But scientists kept getting better and better at accurately measuring distances.  Soon a more precise specification was required.  The laser made it possible to measure the properties of light very precisely.  And Einstein said "the speed of light is always and everywhere the same".  In 1983 scientists took advantage of this to define a Meter as a certain specific number of oscillations of a certain kind of light as measured under certain very specific conditions.  Now a properly equipped laboratory can measure a Meter far more accurately than was possible at any time during the "Meter bar" era.

And this idea of very precisely specifying all the basic units like those of time, weight (actually mass), etc. caught on.  The French developed an entire "Metric" system with seconds (a carry over from the old system), Kilograms (a replacement for the pound), etc.  Now there is a complex system called the "International System of Units".  It is abbreviated as SI based on the French terminology.  It also includes things like Volts, Watts, Ohms, etc. for electricity, Joules, Newtons, etc. for forces and work (to replace things like "pounds force", horsepower, etc.), Celsius (originally Centigrade - to replace Fahrenheit degrees of temperature), and so on.

Returning to the problem with the shape of the earth.  A trick used then and still in use now was to observe a pendulum.  In this case it was used to accurately measure gravity.  If gravity was stronger than normal the pendulum would swing too fast.  If gravity was weaker than normal the pendulum would swing too slow.  This made it possible to measure and map "gravitational anomalies".  We are using instruments that can do the job far more accurately now but the mapping of gravitational anomalies is a booming business these days.  Geologists can tell a lot from gravitational anomalies (i.e. where there's oil) but there are numerous other applications I am going to skip getting into.

Asimov ends this particular portion of the discussion by noting that prior to 1960 the distance between New York and London was only known to within plus or minus a mile.  The techniques I mentioned above (measuring the locations of satellites) was just coming into use as a "by hand" version of GPS.  And at the time a lot of the results of this procedure were classified.  Why?

After the USSR fell in 1989 it turned out that popular maps issued by the Communists showed the locations of their major cities incorrectly.  A typical "error" was say 25 miles.  It was not that they were bad at making accurate maps.  It was thought instead that they had purposely introduced the errors as an attempt to throw off the aim of western ICBM missiles.  Of course, the US had long since switched to the "GPS by hand" method described above, for deciding where to point their ICBMs.

Asimov then moves on to related problems.  If you know the precise shape of the Earth you can accurately calculate its volume.  Then, if you know its weight (or, more correctly mass) you can calculate its density.  But the problem is figuring out its weight.  And here is a good time to explain why scientists use mass instead of weight.

If you stand on a scale what is actually being measured is force.  A certain amount of force bends a spring a certain amount and that can be used to turn a meter a certain distance.  But it is the force that is being measured.  But the force depends on how strong gravity is pulling.  Scientists wanted to get gravity out of the process.  So they decided that matter has an inherent property called "mass".  The force generated in a specific gravitational field depends on the mass and on the strength of the field.  This let scientists split things into a question of the amount of mass, an amount that is independent of what gravity is or isn't doing, and gravitational force, something that is independent of mass and only depends on what is happening with gravity.

If you are standing still on the surface of the earth then weight and mass can seem like pretty much the same thing.  But let's say you are in a car and you haven't fastened your seat belt and your car crashes into a brick wall.  Lots of force is involved and it is likely to get you killed if you aren't extremely lucky.  But this force has nothing to do with gravity.  It has to do with two things.  One of them is how fast you are slammed to a stop (very fast).  The other is your mass.  Remember gravity is not part of the process so "weight" is irrelevant.  But mass is mass is mass.  It can be accelerated by being operated on by the force gravity, which varies depending on altitude, gravitational anomalies, etc.  Or by a car being forced to come to a stop extremely quickly using a process that doesn't involve gravity at all.  By going with mass, which is the same no matter what else is going on (I'm ignoring relativity here) scientists can plug the right number for mass on the one hand and force on the other hand into their calculations and end up with the right result.

Back to the mass (or, for civilians, weight) of the earth.  The problem is that gravity is everywhere.  How do you get outside it so you can measure it?  Newton came up with a formula that looked helpful.  f = ( G * m-1 * m-2) / d**2.  If you knew the value of "f", a force and "d", a distance and if you knew the value of "G", the "gravitational constant" and if you knew the value of m-1 (the mass of one object, say the moon), you could calculate the value of m-2 (the mass of another object, say the earth).  This does not look promising.  We don't seem to know the value of several of those things.  But the formula applies everywhere.  So let's go into the laboratory.  Here we can measure force ("f") using a spring scale.  We can use a ruler to measure distance ("d").  And we can just weigh m-1 and m-2 and use that to calculate the mass of each.  That leaves just "G".  But the formula then lets us calculate its value.  The problem is that "G" turns out to be a very small number.  There is so little gravitational force to measure between two normal objects you can find in a laboratory that it seems impossible to do so.

The first one to make a serious and successful run at the problem was Henry Cavendish.  If we take a thin wire that is say a foot long and fasten one end to the ceiling we can twist the other end and measure the amount of force involved to twist it say 30 degrees.  It's not much but if we use a sensitive spring balance we can measure it.  Now it turns out that if we instead use a 30 foot long piece of the same wire it takes only a thirtieth as much force to twist it the same 30 degrees.  That's the basic idea.

Cavendish took a long piece of very fine wire that was very easy to twist and performed the appropriate measurements on shorter pieces so he could calculate the force necessary to bend it through a relatively large angle like 30 degrees.  Then he put a mirror on it near the bottom and bounced a light off of it from a long ways away (say 50 feet).  This allowed him to measure very small changes in twist.  Then he put two fairly heavy balls on each end of a rod and hooked the rod to the end of the wire.  He made the weights as heavy as he could get away with and he made the rod as long as he could get away with.  By connecting the center of the rod to the wire he could balance everything so that the wire would hold it all up.

Then he took two really big weights.  They could be very large because the thin wire did not need to hold them up.  They could sit on heavy carts on the floor of the laboratory.  He brought each ball very close to one of the hanging balls.  One heavy ball was on the near side of one hanging ball.  The other heavy ball was on the far side of the other hanging ball.  He brought them very close but did not let them touch.  There should be a small gravitational pull between the heavy ball on the floor and its matching ball hanging on the wire.  And there should be a similar gravitational pull pulling in the same direction in the case of the other pair of balls and this should cause the wire to twist.  It did by a very small amount.  But it was enough for Cavendish measure it and to come up with an accurate value for "G".

As another side note:  The University of Washington has been on the forefront of doing these kinds of Cavendish experiments for some time now.  Things get very complicated when you are trying to measure "G" very accurately.  But they have found ways to overcome these complexities. They have succeeded in measured "G" more accurately than anyone else, even themselves in previous experiments, several times now.

So if we know "G" don't we still have a problem?  At this point we know neither m-1 nor m-2 so aren't we still in a pickle?  Theoretically yes but actually no.  What if we put a hundred pound satellite into circular orbit around the earth.  A little calculus (which I am going to skip) tells us what "f" must be.  And we can measure "d".  So that leaves our two "m"s.  But not really.  We can calculate "m-1 * m-2" because we have all the other values in the formula.  But we also know m-1.  It's the mass equivalent of a hundred pounds.  And that leaves only m-2, the mass of the earth, as an unknown.  Plugging all the other numbers in gives us the value of m-2.  If we know the mass of the earth we can go through the same process and get the mass of the moon. We can also use the same process to get the mass of the Sun.  To get a rough number we just ignore the moon and the other planets.  We need to make adjustments for each celestial body's effect to get a more accurate value.

The adjustments can get complicated but astronomers have figured out how to do it so I am going to leave it there.  And we can keep going.  With the mass of the Sun we can calculate the mass of Jupiter or Saturn or, . . .  It's just a matter of using the basic process then applying the necessary adjustments.  The math is complex if you want to get an accurate answer but all we need to know is that it can be done.  We can use the mass of one celestial body to "bootstrap" us to the mass of other celestial bodies.  These techniques certainly work for the planets.  With asteroids there are so many bodies close at hand that in most cases only a rough number can be calculated.  This is also true in some other "many body" problems.  But as computer power increases more and more complex situations can be handled.  Back to Asimov.

He gives us the answer for the density of the Earth.  It is 5.5 times as dense as water, on average.  If we didn't already know, this would allow us to conclude that the Earth is not composed exclusively of pure water.  Okay.  It it of uniform density?  The answer to that question was already known in 1960.  The answer is NO.  How did we know this back in 1960?  From earthquakes.  And that's Asimov's segue into the next chapter.  And that's my cue to end this post.

50 Years of Science - part 5

This is the fifth in a series.  The first one can be found at  http://sigma5.blogspot.com/2012/07/50-years-of-science-part-1.html. Part 2 can be found in the August 2012 section of this blog.  Parts 3 and 4 can be found in the September 2012 section.  Taking the Isaac Asimov book "The Intelligent Man's Guide to the Physical Sciences" as my baseline for the state of science as it was when he wrote the book (1959 - 1960), more than 50 years have now passed but I am going to stick with the original title anyway.  In these posts I am reviewing what he reported and examining what has changed since. For this post I am starting with the chapter Asimov titled "The Windows of the Universe".  This is the last chapter in his "The Universe" section.

Asimov starts the chapter with yet another reference to the 200" Hale telescope situated on Mt. Palomar in California.  That gives me a chance to digress into some telescope basics.  The telescope was invented about 1609 and popularized by Galileo.  Before that astronomical observations were made by eye.  About 3,500 stars are visible in the northern hemisphere with the naked eye.  We now know that this is literally a drop in the ocean compared to the actual number of stars in the sky.  Even in this period it turns out that what was most important to astronomers was the ability to measure angles as accurately as possible.

To address this problem people devised instruments.  These included the quadrant, octant, and later the sextant commonly used by sailors for hundreds of years.  This process culminated with the efforts of Tycho Brahe.  He developed the most precise instruments ever devised to make human eye angle measurements.  He died in 1601 just before the telescope was invented.  His super-precise (for the time) measurements uncovered problems with the Ptolemaic astronomical system that had been in use for centuries.  But before people could digest this and decide on an appropriate response the observations of Galileo started becoming known and they really threw a monkey wrench into the works.  This diverted attention from what Brahe had found but the Brahe measurements ultimately bolstered the anti-Ptolemy side of the argument.  Back to telescopes.

So what's the point?  What does the telescope bring to the table?  The first and most obvious thing a telescope does is magnify things.  This instantly makes it possible to measure angles far more accurately than was possible with the naked eye.  It also makes it possible to view things that are too small to see with just the naked eye.  Galileo discovered the "mountains of the moon".  He observed shadows cast by what appeared to be mountains when observing the edge between the bright and dark parts of the face of the moon.  These features are too small to be seen reliably with the naked eye.  He also was able to make out moons around Jupiter.  Again, these are too small to see with the naked eye.  Galileo saw a lot more things that upset the traditional authorities but that's enough for my purposes.

But being able to see the moons of Jupiter leads to another characteristic of telescopes.  They can make dim things brighter.  It is hard to see the moons of Jupiter not only because they are small but because they are dim.  Galileo was able to easily make them out because his telescope made them brighter.  This is generally referred to as "light gathering power".  The previous property (making things bigger) is generally referred to as "magnification".  And these are the two primary characteristics that the telescope brings to the table.

Now let's take a quick look at telescope design.  The telescope we are most familiar with is the sailor's telescope prominently on display in swashbuckler movies.  It consists of a tube with mirrors at each end.  It does both of the things associated with telescopes.  The lens at the front is bigger than the human eye and gathers more light making the image brighter.  The two lenses work together to magnify the image.  This makes it easy to see the other guy's ship in detail when it is many miles away on the horizon.  And this is the design Galileo used.  But it has a problem.

Lenses work because the material they are made of has a different index of refraction.  All you need to know is that it is a property of things and that it is easily measured.  A vacuum has an index of refraction.  Air has a slightly different index of refraction.  Water has a still different index of refraction as does the glass telescope lenses are made of.  As light passes from material with one index of refraction to material with a different index it bends.  Clever design allows this idea to be turned into a telescope.  So what's the problem?

It turns out that the index of refraction depends on frequency.  So lens glass bends red light through a different angle than it bends blue light.  This isn't much of a problem with a sailor's telescope but it soon became a big problem with the large very precise telescopes that people built to observe the heavens with.  This problem is called "chromatic aberration".  It affects any optical device that passes light through lenses and processes more than one frequency of light.  But there is a solution.  And the man that found it was Isaac Newton.  Talk about genius.

By now the general idea should be obvious.  Don't pass light through lenses.  But how could a telescope be made without lenses?  Mirrors also change the direction light travels.  Curved mirrors can bend light in just the way necessary to make a telescope.  And that's what Newton (yes -- same guy) did.  He put a big for the time (6") mirror at the bottom of a tube.  Then he put a small (compared to the big mirror) flat mirror near the top of the tube.  This mirror was angled at 45 degrees so that the light could come out a hole in the side of the telescope.  That's where he put his eye.  The eye contains a lens but the amount of chromatic aberration is still much smaller than with the old design (called a refracting telescope).  This general class of designs is called a reflecting telescope as light reflects off the mirror.  Pretty much all professional astronomical telescopes now use this reflecting design but with a modification.

A "cassegrain" telescope uses a perpendicular mirror at the front instead of a 45 degree one.  The idea is to bounce the light back down the tube and through a relatively small hole in the center of the primary mirror.  The astronomer's eye (then -- now sophisticated instruments) sits slightly behind the main mirror.  The Hubble Space Telescope, for instance, is a cassegrain telescope.  And for the first 400 years telescopes depended on the naked eye to make observations.  But starting in the nineteenth century then more and more often as the twentieth century advanced photographic plates, often a foot on a side, were substituted.  Since about 1970 the CCD or charged-couple device has been taking over from photographic plates.  CCDs are the electronic equivalent of the photographic plate and are what the Hubble uses.

Now let me circle back to the Hale telescope.  It was the largest in the world for about thirty years.  Why?  Isn't bigger always better.  Maybe not.  The point of a telescope is to make things larger and brighter.  Lets take each in turn starting with magnification.  For a while there was a telescope race as people came up with new designs to increase magnification.  But the race didn't last long.  Remember the "twinkle, twinkle, little star" nursery rhyme?  Stars do actually twinkle.

We now know that this is caused by small irregularities in the air.  Some air is slightly thicker than average and some air is slightly thinner than average.  Winds move these thicker and thinner chunks around continuously.  And it turns out that thicker air has a very slightly different index of refraction than thinner air.  This causes these chunks of air to act like lenses and change the path of light.  This means that sometimes the light from a star is being directed into your eye and sometimes it isn't.  The star twinkles.

It 's kind of pretty when you are gazing fondly at the evening sky.  It is really annoying when you are trying to observe something with a telescope.  The greater the magnification the greater the twinkle effect.  Telescopes were quickly built that were all twinkle and no observation.  There was a limit to the amount of useful magnification a telescope could employ.

But none of this messes with the idea of increasing brightness, right?  You have to work harder but it makes problems here too.  We want to focus all the light of a small star onto a very small point so we get a sharp image of it.  But the twinkle effect moves the location of the star around.  And that results in blur.  And if the star is dim enough you never end up with enough light in any one place to be able to see it at all.  So this means there is a limit to the amount of light amplification that is useful.  The 200" telescope superseded an earlier 100" model.  Doubling the diameter theoretically gave the mirror four times the light gathering ability but in actuality it didn't work four times better.  And there was another problem, gravity.

The mirror needs to have an extremely specific shape.  If not then the light from a star won't all land at exactly the same place.  It turns out polishing the mirror to the extreme level of precision necessary to give it the right shape was relatively easy.  The problem was to make the mirror keep its shape.  To do this it was made extremely stiff.  And that was achieved by using a large, thick piece of glass.  And that made it very heavy.

And as you use the telescope you are moving it around.  And that means gravity is coming at it from one angle now and another angle later.  It worked.  But all the calculations that went into the design said it wouldn't work if you made the mirror a lot bigger, say 400".  And all this weight made the telescope hard to operate.  You needed very big motors to move it around.  And these motors had to position the mirror very accurately or everything was a big waste of time.  So for a long time it looked like 200" was as big as it was practical to go.

But modern telescopes are much bigger.  Astronomers use the metric system so astronomers call the Hale not a 200" telescope but a 5 meter one (200" is almost exactly 5 meters).  There are now lots of telescopes with larger mirrors.  The Keck telescopes in Hawaii have 10 meter mirrors.  The Europeans are currently building a 40 meter telescope and designs have been proposed for even larger ones.  The James Web Space Telescope will house a mirror larger than that of the Hale but in space.  Something obviously changed but what?

The easiest to understand change is that of going from one single main mirror to a main mirror made up of several segments.  Each individual mirror segment is much smaller than the mirror as a whole.  This involved solving the math problem of determining the specific shape each mirror segment needed to be.  Then the harder problem of making mirror segments in these odd shapes needed to be solved but it was.  So what's the advantage of segmenting the mirror?  It means that the stiffness problem is much easier to solve.  One edge of the 200" Hale mirror has to be maintained in a very precise relationship with the other edge.  But if the edges are now say 50" apart this becomes much easier to do.  So the mirror glass can be much thinner and still be stiff enough.  And that saves a lot of weight.

The other design change was to dial way back on the whole "stiff" thing.  If instead of depending on the mirror's built in strength to maintain its shape what if we take very precise measurements, determine what corrections need to be made, and then bend the glass until it is in the proper shape.  Computer power and lasers made it possible to perform the measurements and calculate the corrections to the necessary accuracy.  Then it was a simple process to put a gadget (actually several gadgets) on the back of each mirror segment to bend it the right amount to get the shape right.

It now became important to make the glass thin enough that it could be bent appropriately.  This in turn took a lot more weight out and allowed everything to be lighter and cheaper.  So this new "bend on demand" approach made it possible to have a mirror whose effective size was much larger than before but which still had the right shape.  But what about the twinkle problem?

It turns out that this "bend on demand" capability came to the rescue here too.  With even more computing power (now cheap) it was possible to calculate exactly how much the irregularities in the atmosphere were messing things up.  This made it possible to calculate how to bend the mirror out of what would normally be the "correct" shape just enough to undo the twinkling the atmosphere was causing.  The corrections would have to be calculated and applied frequently (about 100 times per second) but it meant that it was possible to take a much sharper picture of the sky from the bottom of the atmosphere.

This process is called "adaptive optics" and all the big telescopes now have adaptive optics systems.  The last thing you need to make it work is a "guide star".  Measuring it allows the distortions and corrections to be calculated.  The guide star can also be used to verify that the right correction was applied.  If there is a bright star handy close to the portion of the sky you want to point your telescope at then it can be used.  Otherwise, synthetic guide stars can be created using lasers and other tricks.  And with that, let me return to Asimov's book.

Remember that chromatic aberration I was talking about.  And remember the old saw about "turning a problem into an opportunity".  That's the first subject Asimov gets into.  If you introduce a piece of glass into the path of the light from your telescope it will bend the light.  And it will bend the light by different amounts that depend on the frequency of the light.  The piece of glass used for this purpose is called a "prism".  A well designed prism will accentuate this phenomenon.  Why do it?  Because this allows us to study each frequency of say the light from a specific star, separately.  And that allows us to learn a lot about the star.  Studying the various frequencies is called spectroscopy and a device for spreading those frequencies out so that each frequency can be examined individually is called a spectroscope and the resulting pattern a spectrograph.  (And this whole business of studying the spectrum of light also goes back to Isaac Newton.)  So what can we learn from the spectrum of a star?

"White" light will contain all frequencies and the intensity of each frequency will follow a specific pattern.  But real light always has bands that are either brighter or darker than they are supposed to be.  Franhoffer first reported this in 1814.  The dark lanes are "absorption" lines where something has absorbed a particular frequency as the light passes through it.  The brighter lines are "emission" lines.  Something, say a candle, has emitted extra light in particular frequencies.  It didn't take long for scientists to speculate that specific elements caused specific lines.  It turned out that they were right.

Early work identified the new at the time elements of Cesium and Rubidium.  Then in 1868 Helium was identified in the spectrum of sunlight.  Cesium and Rubidium were rare but could be found on earth if you looked hard.  At that time no one had found any Helium anywhere.  (It was later discovered to be a trace component of natural gas and can also be found in even smaller quantities elsewhere.)  The use of spectroscopy to discover solar Helium was a big deal and really put the technique on the map.

These early spectroscopy studies were done "by eye".  The first major step in moving away from this was the invention of the daguerreotype, an early photographic method, in 1839.  As the century progressed photographic techniques improved and photographs became more common in astronomy.  And photography made it possible to use telescopes in a different way.  The standard way is to peer very closely at a small part of the sky.  But this means you can't do a broad "survey" of a larger portion of the sky.  Then in the 1930's Schmidt came up with a telescope design that could do surveys.  But you could not do it with your eye.  You had to use photography.  This was an early example of moving on beyond what the naked eye could show us.

But spectroscopy turned out have very down to earth uses.  Around 1800 Herschel moved a thermometer beyond the visible part of the spectrum.  He detected a warming.  He had discovered infrared light, light with a frequency lower than that of red.  Additional investigation by others led to the discovery of ultraviolet, light with a frequency higher than violet.  The "electromagnetic" spectrum has since been expanded to include (from highest to lowest) gamma rays and x-rays (usually subdivided into hard (higher frequency) and soft (lower frequency)) on the high end and various forms of radio waves (from highest to lowest: microwave, shortwave, and long wave) on the low end.

It turns out gamma rays, x-rays, and infrared waves are all very effectively blocked by our atmosphere.  But radio waves are not.  In 1933 Jansky detected radio waves coming from the sky.  This was the start of what is now a booming field of endeavor, radio astronomy.  Radio astronomy was in its infancy when Asimov wrote his book.  Big dish antennas were just coming into being.  And the now famous giant dish at Arecibo in Puerto Rico was not completed until 1963.  Nor had Lasers been invented yet.  But the predecessor to the Laser, the first Maser had been built.  The idea is the same as that behind the Laser but a Maser uses radio waves and a Laser uses light waves.  It was easier to pull the necessary engineering off with radio waves so the Maser came first by about ten years.  And, although radio astronomers invented the Maser and had done so before the book was written, they were still figuring out how to make the best use of Masers so Asimov does not even mention them.

Another technique that is now in common use is radio interferometry.  This is the process of combining signals from two or more widely separated radio antennas to create a "synthetic aperture" that is as large as the distance between the antennas.  A problem Asimov does address is that fact that some radio wavelengths are very long.  This means you need very large equipment to do anything with them.  The synthetic aperture scheme got around this problem and allowed radio astronomers to make effective use of low frequency (long wavelength) radio waves.  But this came well after Asimov's book.  And this synthetic aperture scheme has now been adapted for use with light telescopes.   There are actually two Keck telescopes in Hawaii.  They can be operated together in such a way as to behave like they are one single telescope with a mirror diameter of 85 meters (almost 300 feet).

One issue Asimov does get into is mapping the Milky Way.  As Asimov put it, "[i]n a sense, the galaxy hardest for us to see is our own."  Radio astronomy has been a big help.  Light is blocked by clouds of dust like the "coal sack" in the southern hemisphere.  But radio waves can penetrate dust easily.  This led to the mapping of the Orion, Perseus, and Sagittarius arms of the Milky Way.  But these arms are only partially mapped and there has not been a lot of progress since Asimov's book.  And the existence of a giant black hole (it weighs millions of times as much as our Sun) in the center of our galaxy was totally unsuspected at that time.  As was the Cosmic Microwave Background, the single biggest discovery in the field of radio astronomy.  It was discovered less than a decade after the book was written.

Asimov does list a number of achievements that had been racked up by radio astronomy.  These include a number of bright radio sources in the sky, the fact that sunspots emit radio waves, the fact that the atmospheres of both Venus and Jupiter are turbulent enough to emit radio waves (as does the cloud surrounding the Crab Nebula), the fact that galaxies collide, and others.  And then there is that workhorse of spectroscopy that was first made use of by radio astronomers, the Doppler effect.  As I have indicated elsewhere, it can be used to measure the speed with which stars (and any other object bright enough to allow a detailed spectrograph to be taken) are moving toward or away from the Earth.  Asimov reports the results of early Doppler work.

At first it might seem like this is a story of scientific results being overturned but that is not so.  In fact, what is on display is a progress toward more and better information.  This does result in some scientific theories being overturned.  But that's why scientific theories are theories.  The possibility always exists that new data will come along that will show them to be wrong in some important way.  But before scientists move on to a new theory they make sure it accounts for all the old observations.

And scientists are pretty good at figuring out how solid their theories are.  Scientists see it as part of their job to theorize before all the data is in.  But they admit it.  They even have a name for this sort of thing.  It's called a WAG, a Wild-Assed Guess.  As more data comes in this might be replaced by a SWAG, a Scientific Wild-Assed Guess.  Neither rises to the level of a "theory", which must have more support.  And some theories are "tentative" while others are "pretty solid".  From there it can move on to being a "well tested" or "foundational" theory or not.  It depends on the data.

And mostly what we see are new discoveries made possible by new and better tools.  The new information does not overturn the old.  It supplements it or "opens new vistas".  Large though it is, there were no tools in 1960 that could have detected the giant black hole at the center of our galaxy.  Scientists knew roughly where the center of our galaxy was ("somewhere in the Sagittarius Constellation") but were the first to admit they knew little to nothing about what was there.  

Obama Bad for Blacks?

The idea that President Obama has been bad for blacks has been popping up recently.  I suspect this is news to people who depend on conservative sources for their news.  But it is now an actual thing in some circles.  And most people would label the proposition preposterous.  But is it?

It certainly is to conservatives.  There he is seen as the unpatriotic President, the anti-white President, the Muslim "president" who was born somewhere else (Kenya, Indonesia, wherever but definitely not in the USA), and, most damningly, THE BLACK PRESIDENT.  Given all this he positively must have been good for blacks, right?

I don't buy any of that (except the "Black President" part - that is unarguably true).  So I was skeptical when I first heard the idea raised.  But there is an actual a case there.  And I have found that when an idea like this appears "out of nowhere" it almost always turns out that it came from a specific somewhere.  I was mystified as to where that somewhere might have been until I saw a mini-review of a book called "Democracy in Black" by Eddie S. Galude, Jr. in The New Yorker.  Mr. Glaude is the head of Princeton's African-American Studies program.  The review indicates that the book contains "a scathing critique of the Obama Presidency" and "describes the 'devastation' suffered by black communities".

I am not positive that this is the one true source for this idea but it is definitely one of them.  That's good enough for the purposes of this post.  So is Mr. Glaude some ivory tower academic with a screw loose, an axe to grind, and a plan to raise his visibility?  I haven't read the book so I can't definitively answer the question.  But my strong suspicion is that he is not.  To understand why let me instead ask a slightly different question:  Have blacks done well during the Obama years?  It turns out that this question is easy to answer and the answer is a resounding NO!

By every measure of economic success blacks are worse off than they were a few years ago.  The number of blacks in the middle class is down.  The percentage of employed blacks is down.  The average income of blacks is down.  And blacks have been hurt more than whites over the period in question.  The middle class as a whole is smaller.  But the black middle class has shrunk more than the white middle class.  Black unemployment rates are much higher than white unemployment rates.  The gap between black income and white income has increased.  Blacks have done badly in both the absolute sense and the relative sense.  Mr. Claude lays the case out in considerable detail (at least that's what the reviews I have read say).

So blacks have done badly under President Obama.  Is it his fault?  There is this theory that the President is responsible for everything, whether good or bad, that happens on his watch.  If you buy this theory then the answer is yes -- President Obama is responsible.  But this theory is inconsistently applied.  To pick just one example:  Jeb Bush has frequently stated that his brother George W. Bush "kept us safe".  Liberals have long said "wait -- what about 9/11" but this observation was ignored until Donald Trump started repeating it.

Lets take a more thorough look at the "blame" question.  Mr. Glaude apportions a generous share of the blame to the President.  Then he moves on to what I feel are more deserving recipients.  But before moving on let's take a deeper look at Obama's role.

Most of the damage was done by the economic crash.  That definitely did not happen on Obama's watch.  It happened on Bush's watch.  This inconvenient truth is ignored by conservatives and Republicans who have invented any number of fanciful excused for why it is actually Obama's fault.  None of them hold water.  But what definitely did happen on Obama's watch was his response.  And the first major action he took was the "stim", his 800+ billion dollar stimulus package.

Conservatives have argued that it was too big.  Liberals have argued that it was too small.  The Obama Administration have argued, rightly I believe, that it was as big as they could make it and still get it through congress.  If the package had been smaller (or not passed at all) it would have resulted in more damage to black economic interests than was the actual result.  So the "stim" was helpful to blacks.  But it was also race neutral.  It was directed neither toward nor away from blacks.  So it was not really an initiative intended to directly help or hurt blacks.  So its effect was best described as neutral.

There was, however, a secondary effect stemming from the "stim".  In my opinion it was poorly constructed.  This was done in what turned out to be a futile effort to attract Republican support.  The "stim" was composed of roughly one third temporary tax cuts (many of which were later made permanent), one third one time subsidies to state and local governments, and one third spending, primarily on infrastructure.  As predicted, the first two thirds were not very effective in stimulating the economy.  But the last third is the part I want to focus on.

Infrastructure projects often include a ribbon cutting ceremony and these ceremonies now commonly feature a giant "check" so that the local TV stations will have a good visual to put on the evening news.  And, of course, there is always a smiling politician standing next to the check taking credit for the funding and, by inference, for his ability to bring home the bacon.  Many of these ceremonies featured Republicans who had voted against the "stim".  And in absolutely every case the Republican in question was careful to ensure that voters were kept ignorant of the fact that the money to pay for the project came from the "stim".  They were thus able to portray themselves as effective at roping in Federal money while simultaneously decrying the hated "stim" as useless spending.  This was one of many tactics Republicans used to rack up big wins in midterm elections in 2010.

And Republicans used the 2010 election win as leverage with which to gut programs that directly and indirectly affected the financial wellbeing of black people.  So by letting the Republicans get away with this trick the "stim" indirectly hurt the economic wellbeing of black people.

Conservatives also railed against Eric Holder, a black man and Obama's first Attorney General.  Holder was supposed to have some kind of diabolical pro-black agenda whose details now escape me.  But Holder spent most of his time in his first few years dealing with the financial crisis.  I can think of no Holder/Obama initiatives on the traditional "war on crime" / "war on drugs" front during this period.

And one thing he did, or more accurately failed to do, was anything effective with respect to Wall Street.  One or perhaps two low level people went to jail and eventually a number of large fines were levied and collected.  But this was seen at the time as largely ineffective and nothing has happened since to change this judgment.   Since Wall Street is almost exclusively a white enclave (the exception being Asians employed in technical rather than leadership positions) an argument can be made that this was anti-black.  But this is another example of an essentially color blind policy where one can argue that a secondary effect hit blacks harder than whites.  But this same secondary effect hit a lot of whites very hard too.

Another example of a color blind policy where it could be argued that blacks suffered disproportionately more harm was in dealing with the "Foreclosure Mess" aspect of the Financial Crisis.  As the economy melted down a lot of people were put into foreclosure.  There is a lot of blame here but I am going to focus on the actual Foreclosure process as that happened almost entirely on Obama's watch.  A lot of just plain bad execution went into the process.  But a lot of criminal behavior was documented too.  There was illegal "robo-signing".  Houses were foreclosed even though the owners were in negotiation with the mortgage holder.  There were even cases where the wrong house was foreclosed on.  But here again few if any were prosecuted and sent to jail.  Many otherwise law abiding and stable black families were swept up in this, more than the standard distributions and statistics would predict.  But the disproportionate impact on black people was a secondary effect.  It was not an intended result of Obama's or Holder's policies.

So in all these cases the effect was unintended and fairly modest.  There were no Obama administration programs that were designed to disadvantage blacks.  And, as I indicated above, the early Holder years included no activity on the crime/drugs front.  In fact the Obama Administration maintained a hard line on the War on Drugs until well into his second term when he finally slightly loosened up on Marijuana.

On the crime front, there was no policy push at all in the first term.  And nothing was done with respect to specific cases like the Trayvon Martin case.  The shooting happened in February of 2012 and the case played out in the months following with essentially no Federal involvement.  At this point we are just short of four years in.  It was only with the Michael Brown shooting in Ferguson Missouri in August of 2014, roughly six years in that the Justice department made any pro-black move.  They eventually became deeply involved.  The Justice department has since been involved in a number of cases.  Blacks are right to fault Obama and the justice department for being somewhere between neutral and hostile when it came to blacks and the criminal justice system.  Claims to the contrary by Republicans are just wrong with respect to roughly the first six years of Obama's time in office.

I have made what I would consider a very weak case for the claim of Glaude and others.  He presumably made a much better one.  And, for the sake of argument, let's assume the case has been convincingly made.  What does Glaude suggest?  He makes what I consider two sensible suggestions and one idiotic one.  The idiotic suggestion is his "blank-out" one.  He suggests that black voters should vote for no one in the 2016 Presidential election.  The idea is to punish Democrats for taking the black vote for granted.  This is a truly idiotic suggestion.

A blank-out vote is effectively a Republican vote.  We have tried the experiment.  Blacks and others turned out in large numbers in 2008 and 2012, the years Obama was running.  In those years, particularly in 2008, Democrats did well.  Did they then work hard to advance a pro-black agenda?  No.  But look at what happened in 2010 and 2014.  Blacks stayed home, they effectively blanked-out in Senatorial, Congressional, and state elections.  And Republicans did very well and picked up lots of seats.

Unlike Democrats or the Obama Administration, Republicans have been actively hostile to a pro-black agenda.  To take the simplest example they have put in draconian voter-ID laws in many states.  These laws are effective at their intended effect, namely denying blacks the vote.  Republicans at the state level (with an assist from the Supreme Court) have often not implemented the Medicaid component of Obamacare.  This has denied poor people (disproportionately black) access to medical care they can afford.  There are other examples.  So recent history has conclusively shown that a blank-out vote by blacks is an anti-black vote.  So what should blacks do instead?

Frankly, they should do what Republicans do and have been doing for a couple of decades now.  And Glaude advocates that in his other two points, the ones I agree with.  One reason Obama has been so late to the party and why what successes he has had have been so modest is a lack of engagement by his supporters.  This includes but is not limited to blacks.  Blacks vote every four years and then they stay home the rest of the time and expect Obama to work miracles all by himself.

Republicans have many powerful, effective, and well organized pressure groups.  The religious right is only one of many.  Karl Rove's ability to mobilize and turn out the religious right in larger numbers than Democrats could imagine is widely credited for Bush's victory over Kerry in 2004.  And this one group has blackmailed the Republican party into its current rabid hostility to abortion.  Abortion is low in most voters' priority list but any Republican officeholder who is not fiercely anti-abortion lives in fear of being primary-ed.

Opinions on abortion have not changed much in recent years.  But the country has made a rapid shift from being strongly anti-gay to being modestly pro-gay.  But being fiercely anti-gay is another litmus test within the GOP.  Very few Republicans who hold office or are contemplating running for office try to buck the prevailing wind on this issue.  Other constituencies within the Republican party guard other issues and agendas fiercely.  And the way they keep their elected officials in line is not by staying home.  They do it by threatening primary challenges.  This does not, at least in theory, jeopardize control of the seat.

But there are few Democratic pressure groups who reliably deliver votes if Democratic candidates toe the line and credibly threaten primary challenges if they don't.  Take the classic black pressure group, the NAACP.  The NAACP almost went out of business in the '90s.  Then Republicans started making anti-black moves like voting restrictions.  This breathed some life back into the NAACP so it is still around.  But it can not turn out large blocks of voters so it has little actual power.  Theoretically there are pro-women's groups, and anti-war groups, and so on.  But they are not as effective as their Republican equivalents.  If they were Republicans would be unable to consistently clean up in off year elections and would not have achieved a near-lock on state government.  Glaude advocates that blacks get way more active and organized and I agree with him.

Glaude also advocates that blacks be more willing to "disturb the peace".  Theoretically, this level of activism is counter productive.  But gays used it effectively.  They went from being invisible to visible to effective in advocating for their agenda.  We have seen the same thing with the Black Lives Matter movement.  There have been people within the black community advocating for years.  But they were not heard.  The "in your face" tactics of many in the Black Lives Matter movement has resulted in attitude and policy changes.  In the years before Black Lives Matter these issues were not a priority of this or previous administrations.  But consistent pressure exerted over a broad front and for a substantial period of time has garnered results.

And there is no doubt that this is a tried and true and effective tactic of conservatives.   We have gotten to the point where gun tragedies are completely routine.  Yet the NRA is famous for its "take no prisoners" tactics.  It is also notorious for its ability to set the legislative agenda.  They have been completely successful at the national level.  They have been only slightly less successful at the state and local level.  The anti-abortion movement has a similar track record.  Clinics have been firebombed.  Employees and patients have been threatened.  Doctors have been murdered.  Yet the NRA and anti-abortion movements are seen as legitimate.  And, more importantly, they are effective.

And this is a broader problem.  Republicans decided at the beginning of the Obama administration to blindly and categorically impede any Obama initiative.  They have been ably supported by their pressure groups.  This has allowed them to maintain this stance for a long time and simultaneously be successful at the ballot box.  Obama has advanced many initiatives that are broadly popular but have gone nowhere.  This is because he has generally been on his own.  He has had little or no active support from pressure groups.  More telling is the fact that he has been routinely subjected to vicious personal attacks.   "He is not Christian."  "He is anti-American".  "He is not an American citizen."  "He is exceeding his authority."  Any similar attack lunched against a Republican from outside the Republican party would result in a fierce counter-attack.  But there is little or no response from outside pressure groups to these attacks on Obama.

There has generally been no sustained and vocal groundswell of support from the constituencies these initiatives are designed to support.  This is true of black issues but also of issues pretty much up and down the line.  Obama is expected to deliver on his own.  An exception is the gay movement.  There has been a lot of outside pressure and Obama has been able to get results.  But this means that in most cases a Republican attack is free of cost.  This has effectively tied Obama's hands behind his back.  It should be no surprise that he has been less effective and slower to the starting line than many constituencies would like.  On the other hand, Republicans in states have been very successful in pushing their agendas.  Their pressure groups have been active and vocal.  Groups that oppose these policies, on the other hand, have been pretty quiet.

We may now be on the brink of more "same old same old".  We are seeing record turnouts for Republican caucuses and primaries.  Blacks (and others) should be shocked and appalled by the promises being made by Trump and the rest of the GOP crowd.  They should be pleased and comforted by the promises being made by Clinton and Sanders.  But people are not turning out on the Democratic side, even for Sanders who has made increased turnout a cornerstone of his campaign.  Sanders people seem to be up for turning out for a rally but not up for showing up on election/caucus day.  Turn out so far on the Democratic side is substantially below 2008 levels.  Blacks and other groups that traditionally support Democrats should be concerned.  If we get a general election that looks more like 2010/2014 than 2008/2012 we will get results very like 2010/2014.  And it will not be pretty for blacks.

The tank was supposed to revolutionize fighting in World War I.  In its first couple of battles it did not.  But it became a game changer when tactics were changed from "tanks can replace infantry" to "tanks and infantry can support each other".  Blacks and others need to vote and to organize and to be in our faces.  Voting is necessary to provide persuadable elected officials.  But people also need to actively participate in the efforts of pressure groups working on their behalf.  This is the tactic of tanks (pressure) plus infantry (voting) that worked in World War I, works for Republicans, and can work for blacks and other constituencies.  We have lived through the political equivalent of "trench warfare" when infantry alone (elected officials) could not get the job done.  Democrats need to go back to the methods Democrats used successfully several of decades ago.  Then elected officials were beholden to active and vocal pressure groups.  Politicians were expected to deliver results but in exchange they could depend on the pressure groups delivering the votes necessary to keep them in office.

Obama has a remarkable record of success given what he has been up against.  Imagine what he could have achieved had he been properly supported.

Digital Privacy

A story has recently broken about a fight between Apple Computer and the FBI.  The context is the San Bernardino massacre which resulted in 14 deaths and many injuries.  The perpetrators, Syed Farook and Tashfeen Malik, were dead within hours.  So the "who" in "who done it" has been known for some time now.  The only open questions have to do with how much help they got and from whom.  There has been a lot of progress on that front too.

Enrique Marquez, a friend and neighbor has been arrested.  Among other things, he purchased some of the guns that were used by the perps.  Literally hundreds of searches have been done and mountains of evidence has been seized.  Online accounts of all kinds have been scrutinized.  Even after all this effort there is more to be learned.  A few days before I wrote this Syed's brother's house was searched.  This was only the latest in a series of searches of his house.

As a result of all this effort the story is pretty much known.  All that is left is to fill in some details.  It is possible that a new major development could be unearthed in the future, say substantial participation by overseas terrorist groups.  But the chances are small.  And that brings us to the phone.

A tiny part of the mountain of seized evidence is a smart phone that belonged to one of the perps.  It has been in FBI custody for some time now.  But that hasn't stopped the FBI from being frustrated, literally.  The phone is encrypted.  The FBI has not been able to break or get around the encryption so they have not been able to access the contents of the phone.  This literal frustration has not been for want of trying.  At least that's the story from both the FBI and Apple Computer.  The FBI has asked Apple for assistance and Apple has provided it.  But the FBI now says Apple must take that assistance to a new level.  And that's what the fight is about.

Before proceeding let me stop to make what I believe is an observation of monumental importance.

ENCRYPTION WORKS

Why is this so important?  Have you seen an "action" movie or TV show any time in say the last 50 years?  These shows often feature a scenario where encrypted data is critical, frequently a matter of life and death.  Sometimes a good guy is trying to decrypt the bad guy's secret plan.  Sometimes it is a bad guy trying to decrypt the good guy's security system so he can steal the secret formula or the invasion plans or whatever.  Regardless, the scene is always handled in the same way.

A geek types away furiously while "action" visuals play out on screen and dramatic music (queue the "Mission Impossible" theme) plays underneath so we will know that important things are happening.  This goes on for about 20 seconds of screen time which may represent perhaps a few hours or days of "mission" time.  But we always have the "Aha" moment when the geek announces that the encryption has been cracked.  And it never takes the geek more than a week to crack it.  In fact it is common for the geek to only need a few minutes.

This is the pop culture foundation for a belief that is widespread and grounded in things that are a lot more solid then a TV script.  We've all seen it over and over so it must be true.  Any encryption system can be broken.  All you need is the genius geek and perhaps a bunch of really cool looking equipment.  People in the real world support this idea often enough for one reason or another that most people have no reason to doubt its veracity.  But it is not true.  And we know it is not true because the FBI has just told us.  Let's look at why.

It starts with the fact that FBI has publicly said that it has been unsuccessful in breaking Apple's encryption.  This is in spite of the fact that they have had weeks in which to try and they have had a considerable amount of cooperation from Apple.  But wait, there's more.  Which government agency is the one with the most skill, equipment, and experience with encryption?  The NSA (National Security Agency).  It's literally what they do. 

Before 9/11 it was possible to believe that the FBI and the NSA did not talk to each other.  It was possible to believe this because it was true.  But in the post-9/11 era those communications barriers were broken down and there is now close cooperation between the two agencies, especially on terrorism cases like this one.  It is literally unbelievable that the FBI has not consulted with the NSA on this problem.  And that means the NSA has also not been able to crack Apple's encryption either.

Let's say they had.  Then the FBI could easily have covered this up by claiming that their own people had cracked the phone.  Even if this was not believed it provides the standard "plausible deniability" that is commonly used in these situations.  It doesn't matter if the official line is credible.  It only matters that there is an official line that officials can pretend to believe.  This is why I believe the NSA failed too.  (For a counter-argument see below).

There is actually a lot of evidence that encryption works but it is the boring stuff that the media ignores.  It gets dismissed as a "dog bites man" story.  I worked in the computer department of a bank for a long time.  They treated computer problems that could screw up data very seriously.  "We are messing with people's money and people take their money very seriously."  I then worked for a company that ran hospitals and clinics.  After observing the culture there I remarked "If you want to see people who treat computer problems seriously, talk to bankers.  They deal with money.  Around here we only deal with life and death and that's not as serious."  That's a cute way of highlighting that people take money very seriously.  And every aspect of handling money now depends critically on encryption.

If even one of the common encryption systems used in the money arena could be cracked there is a lot of money to be made.  Look at the amount of noise generated by people stealing credit card information.  It has finally caused the credit card industry in the US to move from a '60s style magnetic stripe technology to a modern RFID chip based one.  The important take away is that the hackers have never broken into a system by breaking the encryption.  They have used what is generally referred to as a "crib".  One of the most successful cribs goes by the name of Social Engineering.  You call someone up (or email them or whatever) and talk them out of information you are not entitled to like say a high powered user id and password.  You use this information to break into the system.

Important data has been encrypted for many decades now.  The DES standard was developed and implemented in the '70s.  It is considered weak by modern standards but I know of no successful attempt to crack it.  But the long voiced but so far not validated idea that "it might be crackable soon" has been enough to cause everybody to move on.  Something called triple-DES was shown to be harder to crack after double-DES was shown to provide no improvement.  We have since moved on to other encryption standards.

A common one in the computer business is "Secure Sockets".  Any web site with a prefix of HTTPS uses it.  It is now recommended for general use instead of being restricted to use only in "important" situations.  The transition has resulted in some variation of a "show all content" message popping up with annoying frequency.  That's because the web page is linking to a combination of secure (HTTPS) and unsecure (HTTP) web sites.

If the basic algorithm (computer formula) is sound the common trick is to make the key bigger.  DES used a 56 bit key.  The triple-DES algorithm can be used with keys that are as long as 168 bits.  Behind the scenes, HTTPS has been doing the same thing.  Over time the keys it uses have gotten longer and longer.  And a little additional length makes a big difference.  Every additional bit literally doubles the number of keys that need to be tested in a "brute force" (try all the possible combinations) attack.

So piling on the bits fixes everything, right?  No!  It gets back to that crib thing.  Let's say I have somehow gotten hold of your locked cell phone.  What if I call you and say "I'm your mother and I need the key for your phone."  Being a dutiful child you always do what your mother says so you give me the key.  At this point it literally doesn't matter how long the key you use is.  Actually no one would fall for so transparent a ploy but it illustrates the basic idea of Social Engineering.  It boils down to tricking people into giving you information that you can use to get around their security.

If I can get your key I have effectively reduced your key length to zero.  Cribs can be very complex and sophisticated but a good way to think of them is in terms of ways to reduce the effective key length.  If I can find a crib that reduces the effective key length to ten bits that means a brute force attack only needs to try a little over a thousand keys to be guaranteed success.  I once used a brute force approach to figure out the combination of a bicycle lock.  The lock could be set to a thousand different numbers but only one opened it.  It took a couple of hours of trying each possibility in turn but I eventually succeeded in finding that one number.  Under ideal circumstances a computer can try a thousand possibilities in less than a second.

And Apple is well aware of this.  So they added a delay to the process.  It takes about a twelfth of a second to process a key.  This means that no more than a dozen keys can be tried in a second.  And the Apple key is more than ten bits in length.  But wait.  There's more.  After entering a certain number of wrong keys in a row (the number varies with iPhone model and iOS version) the phone locks up.  Under some circumstances the phone will even go so far as to wipe everything clean if too many wrong keys are tried in a row.

The FBI is not releasing the details of what they have tried so far.  And Apple has not released the details of what assistance they have rendered so far.  But this particular iPhone as currently configured is apparently impervious to a brute force attack.  Whatever else the FBI has tried is currently a secret.  So what the FBI is asking from Apple is for changes to the configuration.  Specifically, they want the twelfth second delay removed and they want the "lock up" and "wipe after a number of failed keys" features disabled.  That, according to the FBI, would allow a medium speed brute force attack to be applied.  Some combinations of iPhone and iOS version use relatively short key lengths so this would be an effective approach if the phone in question is one of them.

But Apple rightly characterizes this as a request by the FBI to build a crib into their phones.  Another name for this sort of thing is a "back door".  And we have been down this path before.  In the '90s the NSA released specifications for something called a "Clipper chip".  It was an encryption / decryption chip that appeared to provide a high level of security.  It used an 80 bit key.  That's a lot bigger than the 56 bit key used by DES so that's good, right?  The problem is that the Clipper chip contained a back door that was supposed to allow "authorized security agencies" like the NSA to crack it fairly easily.  The NSA requested that a law be passed mandating exclusive use of the Clipper chip.  After vigorous push back on many fronts the whole thing was dropped a couple of years later without being implemented broadly.

We can also look to various statements made by current and former heads of various intelligence and law enforcement agencies.  The list includes James Clapper (while he was Director of National Intelligence and since), former NSA director Keith Alexander, and others.  They have all railed against encryption unless agencies like theirs are allowed back doors.  Supposedly all kinds of horrible things will happen if these agencies can't read everything terrorists are saying.  But so far there is no hard evidence that these back doors would be very helpful in the fight against terrorism.  What they would be very helpful for is making it easy to invade the privacy of everybody.  Pretty much nothing on the Internet was encrypted in the immediate post-9/11 period.  Reading messages was helpful in some cases but the bad guys quickly learned how to make their messages hard to find and hard to read.

These agencies have swept up massive amounts of cell phone data.  Again, mass data collection has not been shown to be important to thwarting terrorist plots.  After they are on to a specific terrorist then going back and retrospectively reviewing who they have been in contact with has been helpful.  And, by the way, that has already been done in the San Bernardino Massacre case.  But the FBI argues that even after all these other things have been done they still desperately need to read the contents of this one cell phone.  We have been told for more than a decade that the "collect everything" programs are desperately needed and are tremendously effective.  The FBI's current request indicates that they are not all that effective and that means they were never needed as badly as we were told they were.

The FBI also argues that this will be a "one off" situation.  Apple argues that once the tool exists its use will soon become routine.  If cracking a phone is difficult, time consuming, and expensive after the tool exists then the FBI may have a case.  But if it is then what's to stop the FBI from demanding that Apple build a new tool that is easier, quicker and cheaper to use.  Once the first tool has been created the precedent has been set.

The fundamental question here is whether a right to privacy exists.  The fourth amendment states:

The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated, and no Warrants shall issue, but upon probable cause, supported by Oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized.

A plain reading of the language supports the idea that a privacy right exists and that the mass collection of phone records, whether "metadata" or the contents of the actual conversation, is unconstitutional.  The Supreme Court has so far dodged its responsibility by falling back on a "standing" argument.  I think the standing argument (which I am not going to get into) is bogus but I am not a Supreme Court justice.  And the case we are focusing on is clearly covered by the "probable cause . . ." language.  The FBI can and has obtained a search warrant.  The only problem they are having is the practical one of making sense of the data they have seized.

The problem is not with this specific case.  It is with what other use the capability in question might be put to.  We have seen our privacy rights almost completely obliterated in the past couple of decades.   Technology has enabled an unprecedented and overwhelming intrusion into our privacy.  It is possible to listen in on conversations in your home by bouncing a laser off a window.  A small GPS tracking device can be attached to your car in such a way that it is nearly undetectable.  CCTV cameras are popping up everywhere allowing your public movements to be tracked.  Thermal imaging cameras and other technology can tell a lot about what is going on inside your house even if you are not making any noise and they can do this from outside your property line.

And that ignores the fact that we now live in a highly computerized world.  Records of your checking, credit card, and debit card activity, all maintained by computer systems, make your life pretty much an open book.  Google knows where you go on the Internet (and probably what you say in your emails).  And more and more of us run more and more of our lives from our smart phones.  Imagine comparing what you can find out from a smart phone with what you could have found out 200 years ago by rifling through someone's desk (their "papers").  Then a lot of people couldn't read.  So things were done orally.  And financial activity was done in cash so no paper record of most transactions existed.  The idea that the contents of a smartphone should not be covered under "persons, papers, and effects" is ridiculous.  Yet key loggers and other spyware software are available for any and all models of smart phones.

Apple was one of the first companies to recognize this.  They were helped along by several high profile cases where location data, financial data, and other kinds of private data were easily extracted from iPhones.  They decided correctly that the only solution that would be effective would be to encrypt everything and to do so with enough protections that the encryption could not be easily avoided.  The FBI has validated the robustness of their design.

Technology companies have been repeatedly embarrassed in the last few years by revelations that "confidential" data was easily being swept up by security agencies and others.  They too decided that encryption was the way to cut this kind of activity off.  Hence we see the move to secure (HTTPS) web sites and to companies encrypting data as is moves across the Internet from one facility to another.

Security agencies and others don't like this.  It makes it at least hard and possibly impossible to tap into these data streams.  And, according to agency heads this is very dangerous.  But these people are known and documented liars.  And they have a lot of incentive to lie.  It makes the job of their agency easier and it makes it easier for them to amass bureaucratic power.  Finally, given that lying does not put them at risk for criminal sanctions (none of them have even been charged) and can actually enhance their political standing, why wouldn't they?

Here's a theory for the paranoid.  Maybe the FBI/NSA successfully cracked the phone.  But they decided that they could use this case to leverage companies like Apple into building trap doors into their encryption technology.  The Clipper case shows that this sort of thinking exists within these agencies.  And agency heads are known to be liars.  So this theory could be true.  I don't think it is true but I can't prove that I am right.  (I could if agency heads could actually be compelled to tell the truth when testifying under oath to Congress but I don't see that happening any time soon.)  

The issue is at bottom about a trade off.  The idea is that we can have more privacy but be less secure or we can have less privacy but be more secure.  In my opinion, however, the case that we are more secure is weak to nonexistent and the case that we have lost a lot of valuable privacy and are in serious danger of losing even more is strong.  I see the trade off in theory.  But I don't see much evidence that as a practical matter the trade off actually exists in the real world.  Instead I see us giving up privacy and getting nothing, as in no increase in security, back.  In fact, I think our security is diminished as others see us behaving in a sneaky and underhanded way.  That causes good people in the rest of the world to be reluctant to cooperate with us. That reduction in cooperation reduces our security.  So I come down on the side of privacy and support Apple's actions.

In the end I expect some sort of deal will be worked out between the FBI and Apple.   It will probably not be one that I approve of.  It will erode our privacy a little or a lot and I predict that whatever information is eventually extracted from the phone will turn out to be of little or no value. And, as Tim Cook, the CEO of Apple, has stated, once the tool is built it will always exist for the next time and the time after that, ad infinitum.  That is too high a cost.