I Am Afraid I Cannot Value That Home, Dave…

I, Robot.
The Terminator.
The Matrix.
Tron.
2001, A Space Odyssey.
War Games.

What do all of these movies have in common?

Each one, in its own way, is based on some type of artificial intelligence making a play to take over the world.

Initially, it is looking pretty grim for the humans.  The mastermind computer, who sees the human race as a threat, hatches an evil plot to exterminate and/or enslave us all.  Generally, a small band of humans (often led by Will Smith!) manages to find a flaw in the computer’s defenses and, after an epic battle scene, somehow figures out how to hit the proverbial ‘reset’ button thus miraculously saving the human race.  The robots/computers are shut down and go back to being our servants while humans resume being in charge .and all is well again in the 21st/22nd Century.  At least until they film a sequel … but I digress.

With the threat of a computer takeover (hopefully) still a ways off, we should look at how technology is used today and how, in real estate especially, it is being used to replace basic human judgement.  We all recognize the applications of computer technology are almost infinite, but the ability for a computer aided model to interpret massive piles of data and find trends is arguably the most important.   As we improve our ability to have our computers find order amongst chaos, our world should get better/safer/smarter and our lives should improve.  These are all very good things.

While our collective predictive capabilities improve, they are still far from perfect and any prediction without a confidence interval should be discounted.  “It is might rain today” is far less valuable than “there is a 60% chance of rain today” and the “President’s Approval Rating is 49% with a 3% sampling error” means a lot more than “people love the President.”  The confidence interval gives us a sense of how reliable a prediction is and in turn, how much faith we should place in it.

It is unfortunate that Zillow and Trulia have not adopted this practice when they predict value.

First, to clarify…I am not anti-Trulia/Zillow…I am a fan.  I think they are wonderful and extremely important.  I use them constantly.  By typing any address into the menu bar, a user can find out a ton of relevant information about pretty much any property in the world.  These sites aggregate many salient facts into one easy-to-use screen, including how big, how old, when it last sold, what else has nearby has sold, the taxes (and assessment), the schools and neighborhood amenities…these are all a part of the report on the home.  It is comprehensive.

Editors Note – I added this as I needed to amend the original article.  After some research by a friend, with several advanced degrees, who is a staunch advocate of the capabilities of computer modeling, I became aware that these statistics ARE available and they are actually provided by the folks at Zillow.  It requires some reading of the fine print and to click two pages deep, but the statistical accuracy numbers were on the site.  I stand corrected.

To summarize, Zillow makes the following claims:

• 30.3% of the time, the Zestimate was + 5% of the actual sales price
• 53.5% of the time, the Zestimate was + 10% of the actual sales price
• 76.6% of the time, the Zestimate was + 20% of the actual sales price
• 9.1% is the standard error
• These statistics are national.  Most sub-markets are broken down and may be more or less accurate.  Richmond and the surrounding counties seem to fall somewhere in the middle of the pack, collectively.

In effect, Zillow is stating that their estimates for a home sold $300,000 would have been between $270k and 330k about half of the time.

Personally, I had no idea that the Zestimates were actually as INACCURATE as they were. It is a shame how deep this information is buried but I guess I would not want the public to understand this if I was Zillow.

Now back to the story…

While the factual information provided is comprehensive, the estimates of value are not.  I am frustrated by how little Trulia and Zillow go towards debunking the myth that the accuracy of their estimates of value are anything less than excellent.  The public, by and large, places a great deal of faith in the fact that the value estimates provided by these sites should be written in stone.  In doing so, they largely base their behavior on what a computer system says the home is worth without questioning how the value was derived.  While Trulia and Zillow have never come out and claimed that their model is 100% accurate, they vigorously defend their algorithms and actively perpetuate the view that the estimates they provide should be accepted as truth.

I wrote an article about how Trulia and Zillow many times use flawed information to derive their estimates.  The information that they use to plug into their algorithm to determine their ‘estimates’ of value is often times incorrect and in turn, throws off the estimate. The saying ‘Garbage In = Garbage Out’ applies here and incorrect sizes of homes, incomplete inventory and/or other errors is still a challenge to accuracy.  I think it would be more than fair to grant that, over time, as more and more homes sell and thus become part of the database, the factual accuracy of homes will get better.  In theory, as the factual data becomes more and more vetted, the accuracy of the home valuation should improve, right?

Well, as of yet, that is not really the case.

403 Caroline Street in Ashland is a newer home than those around it.  It is near Randolph Macon College and sits on a large private lot where homes change in size and scale rather quickly (as homes tend to do in towns and cities) as you move down the street.

As the screenshot shows, both Trulia and Zillow are picking up on the same size, bedroom and bath count, year built and assessment and both sites have access to the same sold data.  In effect, both algorithms are basing their estimates on the exact same data.

• Trulia values the property at $208,000 (July 11, 2013)
• Zillow values the property at $223,608 (July 11, 2013)

The difference between the two estimates is approximately 7% despite each site basing their conclusion on the exact same data set.  Does 1 + 1 + 1 = 3 or does it equal 4?  How can this be???  Which site is right??  Is either one right??  What went wrong???  The robots are revolting!!!!  Sorry…

The answer is simple…Trulia and Zillow are computer algorithms built by humans (albeit really smart ones who paid attention in their college statistics class) and therefore subjective and imperfect.  If you ask multiple Realtors, you will get estimates that differ as well.  As an aside, I personally would ask $239k right now, but that is because I know that quality properties are at a premium in the Town of Ashland and this home is in a better spot than any of the new homes being built.  But even that is an article for another day.

I know that all of the individuals in my industry get frustrated when we hear a client say ‘Well, Zillow says….’ and then watch as they make decisions based on what one of these sites has set forth as gospel.  I have said this before (and have a feeling I will be saying this as long as I am in this business) – Trulia and Zillow are simply one tool of many that can be used to help value property.  Assessments are also a tool…as is a Realtor Market Analysis (CMA). Hiring an appraiser is a tool in the same way that ‘Trusting Your Gut’ is a tool.  Experience is a tool.  Hearsay is a tool.  Trial and error is a tool as well.

If you live in a neighborhood where 7 houses just like yours have recently sold in 14-21 days of marketing time within 2% of each other, well then you have a good indicator of what your home will sell for.  When you have a home in a neighborhood where the properties are diverse and the sales are sporadic, the ability for a computer (or anyone, for that matter) to accurately model values decreases dramatically.

So the moral of the story is as follows…Trulia and Zillow are wonderful tools but are simply one of many.  Reliance on any one channel as your only source of information is not the recipe for success.  Use the tools you are given but understand what they are.