Cyrpto: Offense or Defense?

Some people have always found it valuable to hide the contents of messages from others.  A common method is Cryptography, or Crypto for short.  Crypto methods date back to the ancient Romans and probably even further back than that.  And for a long time writing was good enough in most cases.  Most people couldn't read so whatever you wrote was safe from the prying eyes of a large percentage of the population.  Only the elite members of society could read so only members of the elites figured into your calculations.

And the two elite groups who were most interested in Crypto were the military and the diplomats.  Both were interested in communicating reliably with their friends while keeping their enemies in the dark.  And this led to a variety of systems.  Simple systems just scrambled the order of the letters or substituted one letter for another.  But by the middle ages the most common method was the Nomenclator.  It consisted of a long list of words or phrases organized into two columns.  The word or phrase in one column replaced the corresponding word or phrase in the other column.  The system was clunky so it was mostly used by diplomats who had embassies that employed code clerks.  The military, who needed systems they could use in the field under combat conditions, pretty much stuck with letter substitution schemes.

The population of people who found Crypto a part of their life got wider with the introduction of the telegraph.  Traveling representatives of companies needed to communicate over long distances and they didn't want competing companies to know what they were up to. So Nomenclators morphed into Telegraphic Codes.  And there was another reason Telegraphic Codes became popular.  They could save money.  The coded message was cheaper to send then the "plain text", the term of art for the original message, because it was shorter.  This got to be a hassle for the telegraph companies so they ended up restricting people to using one of a small number of approved "Commercial Codes".  The telephone eventually doomed all this.

And up to this point all the work was being done by people.  This restricted the options to things people could reliably do in a reasonable amount of time and with a reasonable amount of effort.  That all changed with the introduction of Crypto machines in the 1930's.  The most famous of these is the Enigma machine used by the Nazis during World War II.  Mechanical Crypto machines quickly evolved to become computer based Crypto machines.  But for a long time the use of Crypto was, with the exception of the Telegraphic Commercial codes, restricted to the elites in general and the military and the diplomatic corps in particular.

That all changed when the general public got access to the Internet.  By this time computers were very powerful and capable of implementing very powerful Crypto systems.  And all of a sudden pretty much everybody used Crypto whether they knew it or not.  You care whether your credit card transactions are secure and reliable or not.  And that security and reliability depends critically on Crypto.  Thus endeth the history lesson.

And so far I haven't said a word about the ostensible subject of this post.  Here's where I start.

I am using the words "offensive" and "defensive" the way a military person would use them.  If you are attacking the enemy you have gone on the offensive.  If you are implementing measures to make it more difficult for the enemy to attack you, or for the attack to succeed, you are on the defensive.

So how does this translate into the world of Crypto?  Well, if you are encrypting your messages you are making an attempt to protect them from the other guys.  That is a defensive move.  If you are attempting to decode the other guy's encrypted messages that is an offensive move.  And there is a war going on here.  One side may make a defensive move by deploying a new and hopefully improved Crypto system.  The other side tries to counter this by upping their offensive game.  One side typically has the advantage at any given point.  But the "move - countermove" game goes on and on.  It is commonly referred to in other contexts as an arms race.

I want to get at the question of whether we are striking the appropriate balance between offense and defense.  And this question has been around for a long time.  How much time and effort do you put into developing or enhancing the Crypto systems you use versus attempting to crack the other guy's Crypto systems?  This question was important to ordinary people only at one remove before.  You usually had some investment in some army or another or in some government or another.  So Crypto success for those people you were invested in was a good thing and crypto failure was a bad thing.  Now the impact is more direct.

Recently we had a new computer virus outbreak.  This was different.  It was a "ransomware" attack.  Just like other arms races virus attacks change over time.  Originally a virus attack would wipe out data on your computer.  Then virus attacks evolved into ones that stole data.  Your credit card information (or military and diplomatic secrets) is very valuable if it can be gotten into the right hands.  The value to the attackers of a successful ransomware attack is very direct.  You pay them money.

And the core of the ransomware attack is Crypto.  Your files get encrypted.  Now if this was a movie or TV show at this point we would cut to a shot of one or more people frantically typing, typically onto laptops.  This might be intercut with shots of photogenic arrays of computer screens or of worried people.  All the while dramatic music would be thumping so we would know that something VERY IMPORTANT AND DRAMATIC was happening.  But never fear.  After not very long (we audience members get bored quickly) someone would shout something equivalent to "Eureka".  The Crypto had been cracked and we were all saved.  Happy endings all around.

But in the real world things didn't and don't go that way.  Nobody cracked the virus.  If you didn't send the ransom payment you never would be able to read the files that had been encrypted again.  In short, the offense won and the defense lost.  Why?

Looked at from another perspective this ransomware attack contains some good news.  And the good news is "Crypto works".  (That's something I have noted previously.  See:  http://sigma5.blogspot.com/2016/02/digital-privacy.html).  So if Crypto works and (being the pedantic kind of guy I am I feel the need to repeat myself) it does, then why isn't it used more widely?  And the answer to that question feeds directly into my thesis.

For a very long time the arms of the US government that deal in Crypto have chosen to invest a lot of effort in offensive Crypto and have criminally neglected defensive Crypto.  Governments, including ours, keep deciding it's more fun to crack the other guy's systems than it is to make sure the other guy can't crack their own systems.  They have convinced themselves that their own Crypto systems were unbreakable but that with the proper amount of effort the other guy's systems weren't.  And more and more the arms of the US government have decided that literally any system that is not a US government system is an "other guy" system.

And there is a direct connection between the two.  If everybody is using poor Crypto systems then it is much easier to crack them.  Crypto systems have been cracked going all the way back to the Romans (and probably before).  But somehow the fact that we have succeeded in cracking the other guy's systems (at least some of the time) does not lead to the obvious action of looking hard at our own systems.

There is a trap that governments have been falling into for millennia.  "Our systems can't be cracked".  And there is usually a good reason to believe this.  There is a universal system for cracking Crypto systems.  It is called the "brute force" approach and it consists of trying all the possibilities.  Let's say that it takes a minute to try a possibility, a reasonable figure during the middle ages.  Then if a person lives to be a hundred years old and never stops to eat or sleep they can try about fifty million possibilities in a lifetime.  But let's say our system has a billion possibilities.  Then it can't be cracked using a brute force approach.  It was easy, even a thousand years ago, to come up with a Crypto system that allowed for a billion possibilities.  So these systems were completely secure, right?  Obviously not.

So what's the secret?  The secret is what the British called a "crib", something a student would do to cheat on a test.  The most obvious crib in the Crypto world is to steal the key.  You now have not a billion possibilities to try but one.  But cribs come in lots of different flavors.  Let's say you could find something out or figure something out that reduces the possibilities from a billion to a thousand.  Then the system can be cracked after less than 24 hours' worth of effort.  Cribs that powerful are hard to come by.  But cribs can be combined.  And maybe they only reduce the list to ten thousand or a hundred thousand possibilities.  That's still a big improvement.  Governments tend to assume that they are crib-proof.  But they rarely are.  And the fact that they succeed in developing cribs with which to attack the other guy tends to not have the obvious effect, namely a thorough and careful review of their own Crypto systems.

And the whole Enigma business with Bletchley Park and Magic and all the rest of it is a classic example of this.  Lacking the appropriate cribs it turns out the Enigma machine couldn't be cracked.  Enigma was used by many branches of the Nazi government.  But messages were never cracked for many of those branches.  There is a thing called "Cypher discipline".  This is where you religiously follow all the proper procedures and protocols.  Some Nazi departments were very careful and other departments were sloppy.  But wait, there's more.

Bletchley was a British show but the Americans were heavily involved.  And the Americans ran a parallel operation against the Japanese with considerable success.  Again, some departments of the Japanese government were softer targets than others due in large measure to the degree of adherence to Cypher discipline.  And one of the big beneficiaries of what was cracked was the US Navy.  So did the Navy learn the obvious lesson and make sure they were using good Crypto and good Cypher discipline?  Nope!  The Japanese had a great deal of success cracking US Naval codes and using what they learned effectively.

So has anything changed since World War II?  Yes!  Things have gotten worse.  Various Crypto responsibilities can be found in many parts of the US government.  The NSA, officially the National Security Agency and unofficially "No Such Agency", is a big player in all this.  And the NSA is all offense and no defense.  It turns out that the basic code for the ransomware attack was stolen from the NSA.  It us unclear whether the NSA developed it or just obtained it from elsewhere.  But what they definitely did not do was notify Microsoft of the vulnerability the attack exploited so that a fix could be issued.  Microsoft found out about the vulnerability when leakers posted an NSA list of vulnerabilities and the code that could be used to exploit them on the Internet.  Microsoft immediately issued a fix but a lot of computers were left unprotected for one reason or another.

But wait, there's more.  As I indicated above, there are lots of ways to do Crypto.  For decades the NSA has seen it as their right to decide which systems people can use.  And they want those systems to be easy for them to crack.  Then some civilians came up with a system called RSA, which turns out to be completely secure if no cribs are handy.  And this was a Crypto system that the NSA could not control.  This forced the NSA to respond by issuing a pretty good Crypto system called DES.  But we wouldn't have DES if we hadn't had RSA first.

And this policy of doing their best to keep good Crypto out of the hands of anybody but the US government has been a long standing policy of the US government with the NSA often taking the lead.  A couple of decades ago the "Clipper" computer chip was announced.  All computes were supposed to use a Clipper chip to do their Crypto.  But the Clipper came with a back door that the NSA, the FBI, and other government agencies could use.  Fortunately, that proposal died quickly.

9/11 produced the USA Patriot Act.  It in turn produced the most complete gag order in history.  Agencies like the NSA and the FBI can ask you for any kind of data they want and you are forbidden from even disclosing that a request had been made.  Companies like Google and the mobile phone companies were ordered to disgorge vast amounts of data about literally everyone.  At the same time they were forbidden from even telling anyone about the existence of the order let alone its contents.  This was all revealed by Edward Snowden.  The Snowden revelations have caused these kinds of provisions to be dialed back but only to a modest extent.  The main provisions are still in effect.

The FBI was in the news a few months back because they were asking Apple to hack their own phones.  This is because newer versions of the iPhone use better and better Crypto to effectively keep the data on them private.  Various government agencies, including but not limited to the FBI and the NSA, have repeatedly asked for legislation mandating back doors into consumer devices like phones.  They have also asked for back doors into data centers run by Google, mobile phone companies, and others.

There is an obvious value in letting the appropriate agencies in the appropriate circumstances get access to the appropriate data.  But it's the whole "appropriate" thing that is the problem.  It turns out that you can't draw a bright line indicating where the boundary between appropriate and inappropriate should be.  And even if you could the boundary is not a real boundary.  If the appropriate agencies can get appropriate access then inappropriate agencies will also be able to get inappropriate access.

The news has been littered with these stories for the past few years.  Credit card data gets stolen so routinely that it now hardly qualifies as news.  And if the NSA can get into Iranian computers the North Koreans can get into the computes at Sony Pictures studio.  And Russian hackers can get into the computers of the US State Department, campaign committees belonging to both the Democrats and the Republicans, and so on.  Apparently the only place they couldn't get into was Hillary Clinton's home email server.

These systems could be much more secure.  But various US government agencies have been doing what they can to keep them insecure.  It is beneficial to these agencies for them to be able to get into the systems of other countries.  But the cost is great because it means that our systems are vulnerable to other governments like Russia, China, and even the likes of Iran and North Korea.  They are also vulnerable to criminals both domestic and international.  It even means that our systems are vulnerable to amateurs interested in celebrity sex tapes, gossip, and the like.  It's gotten to the point where even some kid who wants to cyberstalk another kid can break into a surprising number of places.

All of this is the cost of the policy pursued by so many in the government of keeping our online systems vulnerable.  And the big problem is it is an unacknowledged cost.  It affects us all in ways we notice and ways we don't.  Is the benefit really worth the cost?  I don't think so.  Reasonable people may disagree with me.   But the big problem is that almost nobody knows that this tradeoff is being made on out behalf.  So they don't even know that it is a question that needs to be investigated.

Residential Real Estate

This is one of those subjects that I like to tackle.  Why?  Because there is so much nonsense and half baked analysis out there.  Straightening all that out is one of the reasons I write this blog.

I live in Seattle.  If you are part of the government of a city this is one of those "I wish I had that problem" problems.  Real Estate prices in Seattle are literally skyrocketing.  For several years now prices in Seattle have gone up more quickly than pretty much anywhere else.  The well respected Case Shiller index has rated Seattle as the place where prices have gone up the fastest for several months in a row now.  It has now been going on long enough that people are shouting "something has to be done".  But what?

Rather than doing my usual historical backgrounder at this point I am going to do a "fundamentals" backgrounder.

Residential real estate is a "market".  It is the very thing economists are talking about when they talk about a markets.  As such it is subject to the rules of supply and demand.  In an unconstrained market (more about this later) if there is more demand than supply prices go up.  If there is more supply than demand prices go down.  And that's what has been happening in Seattle.  Little new supply has come online.  Meanwhile Amazon, the web retailer, has been hiring like mad for jobs in buildings located in Seattle.  A lot of these are high paying jobs.  So these new hires have money in their pocket and they are looking for a convenient place to live.

And it's not just Amazon.  Business in Seattle is doing well.  So employment has been skyrocketing.  That in turn has driven up demand for housing.  That in turn has driven up prices.  And so far that has not resulted in a bunch of new housing getting built.  And that takes us to the whole "unconstrained market" thing.

Consider Houston, Texas for a moment.  Houston is in the middle of nowhere.  And by "nowhere" I mean it's surrounded by cheap flat land as far as the eye can see.  If Houston needs bare dirt on which to build houses it just annexes a big chunk of land adjacent to city limits and tell developers to get on with it.  And Houston has essentially no zoning laws.  Developers cna build pretty much anything they want and they do.  Houston has been growing very quickly for a long time now.  But housing is still cheap.  And this is because increased supply keeps up with increased demand.  Houston is a classic example of an unconstrained market.  And it demonstrates how supply and demand works in one situation.

Let's look at another situation, Detroit, Michigan.  Detroit is a big place.  It sprawls over 140 square miles.  That's big.  And back in the day Detroit had a large enough population to fill all that land up.  But the auto industry declined.  And production moved to the south.  And automation drastically reduced the number of people it took to build a car.  So good paying jobs went away.  And eventually so did the population.  Detroit has plenty of supply.  You can literally pick up a nice house for $10,000.  But there is no demand.  People with no jobs and no other source of income can't afford a house even if it only costs $10,000.  So a plentiful supply coupled with a total lack of demand has driven housing prices in Detroit effectively to zero.

There is something else going on here, something mostly ignored by economists.  That is the rate at which the market can respond to supply/demand pressures.  Economists generally look at the situation as a static one.  They can use the supply/demand curve to calculate what direction the market is being pressured to move toward.  But they generally ignore the rate at which the market can respond.  If, for instance, we are talking stock prices then things can move quickly.  With high volume computerized trading large moves can happen in less than a second.  That is not true with the housing market.  It takes years for the market to respond.

Detroit grew relatively slowly from the start of the twentieth century to roughly the middle.  There was a steady growth in jobs.  So there was a steady demand for more housing.  Developers could see not only the current state of the market (favorable toward additional construction) but also the trend (also favorable to additional construction).  So the city annexed land and developers developed it and Detroit has a housing market that stayed in balance.

But in the last few decades Detroit has suffered shock after shock.  And they have turned the pressure for housing from positive (build more) to negative (we already have too much).  And while the methods for adding housing are well understood by all the players (cities, developers, consumers) the methods for reducing housing are not well understood by the same set of players.  So nobody has responded well.  The city has gone bankrupt.  Developers have either gone broke or moved somewhere else.  Consumers have been stuck with houses they can't afford and can't sell.  Lots of Detroit housing stock has been foreclosed on or abandoned.  It has turned from an asset into a blight. 

But hope springs eternal.  So the sensible idea of bulldozing the dangerous derelicts and consolidating the city down to a size more consistent with the actual population, a move that would save the city tons of money by reducing the footprint of utilities, street maintenance, police/fire, bus service, etc. has, in the short run, proved impossible.  Detroit and Seattle are opposites.  The market has changed drastically.  In Seattle's case it is for the good and in Detroit's case it is for the bad.  But in both cases the market has not been able to change quickly enough to put things back in balance.  This fundamental unwillingness to understand that the problem in both cases is a slow response to changed market conditions has generated a lot of anguish in both cases.

But back to the whole "unconstrained market" thing.  It has been more than 50 years since Seattle has been able to add undeveloped land by annexation.  It is now too late.  Seattle is surrounded on all sides now.  It is bounded by water on two sides and by other municipalities on the other two sides.  Even if it succeeded in annexing land that annexation would not help.  Houston can still annex undeveloped land but all the land surrounding Seattle is already developed. So the Houston solution is unavailable to Seattle.  Here's where math kicks in.

It's all about density.  If the population goes up the fact that Seattle can't get any bigger means the density goes up.  It really is that simple.  So one solution is to freeze or decrease the population.  But before you do that it might be a good idea to ask Detroit how that worked for them.  And in fact, Seattle's population actually stayed nearly constant from 1960 to 1990.  That means the housing stock stayed the same, right?  Well, actually no.  By 1960 Seattle had pretty much developed all the land that existed within its city limits.  Oh, there was the odd lot here and the odd lot there. But there were no large tracts of undeveloped land.  And that means that construction of your standard stand alone house on its own lot contributed little or nothing to the increase in housing stock.  But housing stock did grow.

Year by year multifamily development took place.  This was a mix of apartments, condominiums, and town houses.  So pretty much every year the number of square feet of space available for residential use went up.  And mathematics demands that the average resident kept consuming more space on average.  In 1960 most houses were occupied by families.  So two, three, four, perhaps more people occupied each house.  By 1990 the average number of people in a house had declined a lot.  I live alone in a house that that in the past has hosted between two and four people.

There are now many houses with one or two people in them.  This large decrease in the total number of people living in houses has been balanced by a large increase in the number of people living in apartments, condominiums, and town houses.  The two changes, the decrease in the population living in stand alone houses and the increase in the number of people living in multiple family dwellings, pretty much exactly balanced each other out.

At least that was the story through 1990.  The 2000 census showed for the first time in a long time a significant increase in Seattle's population.  The 2010 census showed a big jump.  And all the numbers say that since 2010 Seattle's population has been skyrocketing.  For a couple of generations the Seattle political establishment operated in an environment where the population of Seattle was very stable.  They had no experience with what to do when things changed and the population started to grow and then started to explode.

Standard market analysis tells us what the solution is.  If demand is growing the proper response is to increase supply.  And, as noted above, adding undeveloped land and then developing it (the Houston solution) is not an option.  And, assuming we take the "drive the population down" option off the table, the only option is to increase density.  And that means more apartments, condominiums, and town houses.  It's really that simple.

But that clashes with Seattle's vision of itself.  Seattle sees itself as some kind of suburb where single family houses on generous lots sprawl as far as the eye can see.  So a few years ago Seattle put in an ordinance limiting the heights of residential buildings and constraining where they could be built.  "Multifamily is just not Seattle."  And it worked in that far too little multifamily residential development has taken place.  And that's why supply and demand are currently so far out of whack.

And so we are back at it.  The Seattle City Council has recently been raising allowable building heights in several neighborhoods.  This has created lots of unhappy people.  "You are destroying the neighborhood."  But it doesn't matter how good or bad a neighborhood's ambience is if you can't afford to live there.  And that's what is happening.

People are getting priced out.  Renters have seen rents skyrocket.  I have already mentioned that home prices have shot up so more and more people are being priced out of the market.  So we are not talking about whether things are going to change.  They are.  It's inevitable.  We are talking about how things are going to change.  Pretty much nobody has figured that out.  So a lot of what I hear boils down to "I want things to stay the same."  That's not an option.

So among the actual alternatives how do we want things to change?  Take increased density as a given.  We are going to see more apartments, condominiums, and town houses.  And the most basic question revolves around economics.  How expensive do we want these units to be?  There is a lot of hew and cry that Seattle needs more low income housing.  Say that is so for the sake of argument.  How do we proceed?

Well the consensus among low income housing advocates, and they are a large and well organized group in Seattle, is that developers should be required to create lots of units that are priced below market so that low income people can afford them.  This is a variation on rent control.  The rent on some units is artificially controlled to be below market.  You can argue about the specifics for decades, and people have, but that's the basic idea.  So right now in Seattle certain projects are required to include a certain number of these units.  But the demand always vastly exceeds the supply.  So a big bureaucracy must be put in place to decide who gets to actually occupy the units.  And there are always lots of deserving people who don't make the cut.  What do you do with them?  Nobody's figured that out yet.

And there is lots of experience with rent control.  New York City started out doing it a long time ago on a small scale.  Then since demand always vastly exceeded supply they kept expanding the program and making it more bureaucratic and baroque.  And it never quite worked.  And it produced the New York of the '70s.  You had large numbers of buildings that were falling down because they were badly maintained.  Landlords decided that it was better to let the buildings fall apart than maintain them.  Why?  Rent control.

And for one reason or another there was a benefit to the owners to frequently selling the buildings so they did.  The result was a long term downward spiral in the condition of a very large amount of what had initially been perfectly good housing stock.  New York has gotten rid of a lot of its rent control and lots of new residential construction has been the result.  But they haven't figured how to get rid of the last vestiges of rent control.  So it limps on.  And it's not like they figured out how to do rent control right somewhere else.  The New York experience is particularly extreme but it hasn't really worked anywhere else either.

You can probably tell I am not a fan of rent control.  So what's the alternative?  It is a variation on the Houston plan.  If developers can make money developing inexpensive housing that relatively poor people can afford they will.  But the profit margin on expensive housing is higher.  So builders will only develop inexpensive housing if the market for expensive housing is already saturated.  That is definitely not true in Seattle.  And that's reflected in the fact that there is a lot of housing going up.  But it is all aimed at the high end of the market.  Advocates are correct in the short run in saying that developers are tearing down relatively inexpensive housing in order to build expensive housing and that's making the problem for poor people worse.  But I suggest taking a longer view.

Lots of the people moving into the new expensive housing are currently living in less expensive housing.  As they move up it will free up mid market housing.  Now the market in Seattle is so hot that this will not help much in the short run.  But short run thinking is what got us where we are now.  Developers always overbuild if given the chance.  So the first thing that needs to be done is to give them a chance.  The amount of housing coming on line this year (2017) and next will set records.  That should depress prices.  That is unless the market is so out of balance that this massive amount of additional supply still doesn't put us in balance.  And in Seattle's case that is a significant possibility.

But that just means that the developers of the current slate of projects will make a ton of money.  And that will just encourage more and more developers to build more and more projects.  At some point they will get ahead of themselves.  That is if they are allowed to.  And the city's recent actions of increasing allowable building heights is a step in the right direction.  The first step is allowing developers to develop.  That is pretty straight forward.

The next step is much harder.  Once they have saturated the expensive end of the market they will look down market toward the inexpensive part of the market.  And a lot of crap inexpensive housing has been built at one time or another in one place or another.  Housing advocates are almost as worried about this as they are about pricing people out of the market.  So they tend to react by advocating for complex zoning whose objective is to force developers to build "nice" projects.  The problem is no one has figured out how to write zoning rules that mix nice with inexpensive.  Lots of zoning rules promote expensive.  But they have a poor track record of promoting inexpensive buildings that are also nice.

My brother is an expert in this sort of thing.  He hasn't figured out how to do it.  He argues that it is possible to make a development both nice and inexpensive.  He can show you a whole bunch of examples of this being done.  But he hasn't figured out how to write zoning or other codes that makes it happen.  And neither has anybody else.  My recommendation is one of those that no one will like.  I think zoning should focus on safety and that sort of thing.  If developers want to build ugly buildings, let them.

And I think my recommendation will eventually fix Seattle's problem.  If developers build enough supply then eventually it will outrun demand.  At that point prices will moderate.  But the soonest I can see my recommendation fixing the problem is five years and I have to admit it might take longer.  And no one wants to wait that long.  So we will probably screw things up in Seattle by doing some kind of idiotic variation on rent control.  That will discourage developers from developing and put off the day when supply outstrips demand and prices moderate.

So I fear we are in for a lot more nonsense and half baked on this subject.  Oh, well.