UpstreamRE
  • Updates
  • FAQ
  • Sign Up
  • About Us
    • Board of Managers
    • Our History
    • Contact Us
    • Media >
      • Downloadable Documents
    • Events
    • Terms
    • Privacy

Swimming Upstream

Before You Storm the Castle… An Upstream / RESO Purity Check

10/24/2016

8 Comments

 
Picture
"Liar! LIAR! LI-A-A-AR!" says Valerie to Miracle Max.  I apologize for the Princess Bride reference, but that shrill phrase is often what I hear whenever I use RESO standards and Upstream in the same sentence.  It's understandable… there is always a tension between Theory and Practice. 
 
To explain this relationship by way of practice, Theory is abstracted Practice, and Practice is applied Theory. Unfortunately, those who specialize in practice often claim that theorists are detached from the ‘real world,’ and those who specialize in theory often claim practitioners have no fundamental understanding of what they do.  
 
The Upstream/RPR team is the best hybrid.  One that embodies the abstraction/application process. We hope to be a conduit, bridging the two worlds. RESO standards have long been an established fact in our industry.  They offer the most efficient path to publishing listings for product integrations, and the compliance deadlines that the industry has given itself have been almost universally met.  

However, a project like Upstream both leverages the industry’s broad adoption of RESO standards, as well as pushes the boundaries of how the standard is deployed today towards the many different areas into which it will expand in the future.  For that, we need to get into some theory, and explaining core theory to a practitioner is no less an art form than combining the essences of practice to those who theorize. Building this bridge is an act of creation… of building.  
 
There have been many questions, even accusations, regarding Upstream's adherence to RESO standards.  As we approach the RESO 2016 Fall Conference, now seems a good time to address these concerns “head on.”
NOTE:  Now for my second apology… the remaining content "gets in the weeds" a little.   I'll do my best to keep it entertaining.
Questions about Upstream and RESO standards can be grouped into four topic areas: standard names, RETS layout, business rules and the Web API.  In all cases, the Upstream team has a strong, contributing relationship with RESO and we embrace standards whether they are real estate, programming, or other.  Our accelerated use cases (practice) sometimes push the edges of the standards as they are currently defined.  Our commitment was, and remains, to adapt with feedback and contribute those learnings in the hope to contribute what we learn and help to expand the standard.  Now, get out your machete… we're going into the weeds.

Standard Names
 
All listing input, listing data storage, and listing delivery in Upstream is natively implemented using RESO standard names from the Data Dictionary.  Upstream and RPR implemented its initial database schema and API endpoints (including collections and field names) initially using RESO Data Dictionary 1.4 and later updated to match version 1.5 in the fall of 2016.
 
Our pilot only extends to five MLSs, which include fields that are not currently the Data Dictionary.  In those cases, we use a naming convention and schema that can be changed as the Dictionary evolves.  Additionally, Upstream is parcel-centric vs. listing-centric.  Updating listings from a parcel-centric database introduces new dimensions to listing characteristics and enumerations from assessment and other sources of property data, and business rules.  As we discover and document these nuances, we contribute our findings to accelerate the knowledge base for expanding the Dictionary.
 
So why are there questions about standard names?   Why can't Upstream pass field level compliance?  It's a case for… well, case.   The RESO Certification requirements state:  
"The applicant’s field name  MUST  identically match the Data Dictionary Standard Name when the definitions match."
RESO standard names leverage XML-world, PascalCasing conventions.  My intuition is the original RETS layout lived in an XML world.  However, the development community, and RESO, has evolved and adopted the JSON format.  It is now the most common data format used for asynchronous browser/server communication.  Upstream fully embraces JSON with Google's JSON Style Guide which states names must be camelCased, ASCII strings.  Simply put, Iphone becomes iPhone."  Same letters, different casing.   RESO has fully adopted JSON as a data format.  Upstream continued and adopted the JSON naming convention hoping to "skate where the puck is going."  Unfortunately, it's not an identical match which translates, on a technical level, to a failure of field level compliance.
 
RETS Layout
 
The Upstream/RPR team initially proposed a delivery layout that modified the standard RETS format used for listing distribution.  We supplemented it with enumerated collections to make it easier for developers to access enumerations (status variants, historical updates, etc.) in a single pass, and that also better expresses the parcel-centric design (and data richness) of the data architecture. 
 
While some developers appreciated the extension, some argued that the classic RETS format is more familiar and universally accepted.  Well… that's good feedback, so Upstream will offer a flat, RETS layout as an alternative option, using the RESO Dictionary Platinum format.  My intuition is the format will eventually uncover issues once the Web API paradigm shifts from "syndicating" listings to "updating" listings (see Web API).  When we get there, we'll cross that bridge together.
 
Business Rules
 
Many believe that incorporating the business rules across hundreds of MLSs is simply impossible.   And we've discovered that each MLS has hundreds if not thousands of rules. The good news, we've already found patterns.  Although the variables change, the thematic rules are often similar.   We document every rule we discover and hope to publish anonymous, best practices from around the industry.  With that said, RPR and Upstream continue to contribute to the evolution of a business rules standard in the RESO R&D workgroup.  We're excited that RESO has created an ecosystem that can distil broad industry input and create a framework for efficiency while allowing local nuances.
 
Web API
 
As mentioned earlier, the RETS layout and the current Web API definition works well germane to MLS fields and when focused on syndication (distribution).  Similar to Upstream's goals, RESO and the Data Dictionary are expanding beyond MLS fields (as evidenced by the Contact standard) and evolving from simple syndication to “Update.”   Updating presents interesting challenges not seen in today's distribution paradigm.   Let's look at mapping into the MLS vs. mapping from the MLS.  
 
MLSs may have many variants of statuses that get mapped down in a RETS feed for aggregators.  For listing distribution, this is easy:
Picture
​But what happens when starting from listing input using the RESO status like Upstream does, and going the other way?  How does the MLS decide?
Picture
​And then what happens when creating your listing in two MLSs?
Picture
Designating a listing in a RESO status alone may not be enough – other attributes will be needed to indicate the intended status in the MLS.  Similar challenges must be resolved in enumerations: 

  • A water source of “Private” in Upstream listing entry using the Dictionary could be “Private Co-op” or “Private/Mutual” in one of our pilot MLSs
  •  A water source of “Public” could be “City Water” or “MUD Water.”
 
Upstream creates the need to go faster down the path towards "Update" that RESO was already heading.  We hope that our initiative creates an excellent feedback loop into RESO as we work through these questions with our pilots and subsequent expansion.  
 
Whether it's board participation by various Upstream Board of Managers or workgroup participation by Upstream's partner RPR, we see this as a partnership.  So, if you still plan on storming the castle, put the machete down and bring good wine.   We have long, interesting conversations about the future ahead of us.
8 Comments
Maryann Pearson
10/25/2016 10:05:44

Maybe I'm missing something, but shouldn't it be relatively simple for the MLS providers to provide an auto translation in their systems so the RESO verbiage input fits the local MLS category (if they insist on maintaining it/them)..the "greater good" is served w RESO..

Reply
Alex Lange
10/25/2016 11:53:29

Hi Maryann,

It can be simple if one assumes a one to one relationship between a listing and the MLS. I’ve had many industry professionals question storing the MLS version vs. the RESO version. Since Upstream is parcel-centric, a home is represented by a single record even if it’s in three MLSs. The more difficult puzzle to solve is which MLS variant is stored? It’s uncommon knowledge that 10% of individual broker offices (not brokerages… each office) spans more than one MLS. If the variants are different, which one do you store? What business rules win?

As far as the unification of standard enumerated values, I think we’re a long way from those standards especially germane to listing status. Localized rules impact contracts and offers based on granularity. In many locations, you can still show a house under RESO “Under Contract” status depending on the state of an offer. Those details make a difference. We will get there… it’s these nuances Upstream is trying to ferret out for the standards groups to consider.

Reply
Maryann pearson link
10/28/2016 00:44:06

I see your point, but think we tend to over complicate the differences. A property "in contract" either can be shown to other prospective buyers, or cannot. Verbiage may differ - contingent/show, in contract, pending/show, pending/do not show, etc. But the intent can be deciphered: is there an accepted offer? Yes/no, can the property still
be shown? Yes/no, is the contract firm? I.e. No contingencies? Yes/no?
Rather than using the MLS proprietory language, the input could be posed as these questions/options, then translated to an agreed set of "terminology". I think not that not difficult to achieve. X=y

Rob Larson
10/28/2016 16:03:29

The RESO dictionary provides the Standard Status as an option to summarize your more granular MLS Statuses. The Dictionary also provides the MLS Status field so you can map your more detailed MLS Statuses and remain compliant. So the above examples would never need happen.

Participation in the Data Dictionary group, or posting questions to ddwiki.reso.org are great ways to clear up misunderstandings like those in this article.

Reply
Alex Lange
10/28/2016 16:43:59

I understand your argument. Help me see how to solve the “one to many” issue especially when variants in the “many” allow different behaviors for the “one.” My intuition is there’s an assumption that if 123 Main Street is listed in two MLSs, two records exist so you can map the RESO standard to the internal MLS value. In a parcel-centric schema, there is only one record for 123 Main Street. When it becomes a listing in two MLSs, those are enumerated events. Those events can allow different behaviors like if the house can continue to be shown even if under contract (active contingent vs. bumpable buyer example). When you try and distribute the parcel record in a “flat” format, who wins? Sending enumerations matter…

Reply
Rob Larson
10/28/2016 17:07:01

If you understand that you would never map Standard Status to MLS Status; and that in your example you would map MLS Status to MLS Status, you're on he right track. Feel free to contact me to discuss your property centric structure. I'm happy to help.

Reply
Rob Larson
10/28/2016 18:24:43

Here's an article I wrote the other day to assist members with the status concepts . You can access with your RESO logon, or if desired I can post somewhere that doesn't require a login.

http://members.reso.org/pages/viewpage.action?pageId=30446252

Reply
Alex Lange
10/29/2016 08:13:41

Cool. I’ll set up some time after NAR Annual for a brainstorming session. There're all kinds of fun puzzles to think through such as pricing history, parcel attributes not considered in the listing world (like assessor data), how events like taking a listing off the market and then putting it back on should really be handled (especially for internal brokerage systems), etc.

I’ll come to you and more importantly, bring wine!




Leave a Reply.

    Send Me Updates!

    * indicates required

    Categories

    All
    Opinion
    Q&A
    Quotes
    Updates

    Archives

    December 2020
    August 2020
    October 2019
    September 2019
    July 2019
    June 2019
    February 2019
    December 2018
    October 2018
    August 2018
    June 2018
    May 2018
    April 2018
    March 2018
    February 2018
    November 2017
    August 2017
    May 2017
    April 2017
    March 2017
    February 2017
    January 2017
    December 2016
    October 2016
    September 2016
    August 2016
    July 2016

    RSS Feed

    Our Goals

    We seek to create efficiencies by reducing redundant entry, help brokers distribute deliberately by giving them control over who can access their data and inspire innovation by removing artificial barriers when the broker allows.

Privacy Policy    |    Terms & Conditions
Copyright © 2019 UpstreamRE, LLC.  All Rights Reserved.
  • Updates
  • FAQ
  • Sign Up
  • About Us
    • Board of Managers
    • Our History
    • Contact Us
    • Media >
      • Downloadable Documents
    • Events
    • Terms
    • Privacy