Before You Storm the Castle… An Upstream / RESO Purity Check

"Liar! LIAR! LI-A-A-AR!" says Valerie to Miracle Max.  I apologize for the Princess Bride reference, but that shrill phrase is often what I hear whenever I use RESO standards and Upstream in the same sentence.  It's understandable… there is always a tension between Theory and Practice. 
 
To explain this relationship by way of practice, Theory is abstracted Practice, and Practice is applied Theory. Unfortunately, those who specialize in practice often claim that theorists are detached from the ‘real world,’ and those who specialize in theory often claim practitioners have no fundamental understanding of what they do.  
 
The Upstream/RPR team is the best hybrid.  One that embodies the abstraction/application process. We hope to be a conduit, bridging the two worlds. RESO standards have long been an established fact in our industry.  They offer the most efficient path to publishing listings for product integrations, and the compliance deadlines that the industry has given itself have been almost universally met.  

However, a project like Upstream both leverages the industry’s broad adoption of RESO standards, as well as pushes the boundaries of how the standard is deployed today towards the many different areas into which it will expand in the future.  For that, we need to get into some theory, and explaining core theory to a practitioner is no less an art form than combining the essences of practice to those who theorize. Building this bridge is an act of creation… of building.  
 
There have been many questions, even accusations, regarding Upstream's adherence to RESO standards.  As we approach the RESO 2016 Fall Conference, now seems a good time to address these concerns “head on.”

NOTE:  Now for my second apology… the remaining content "gets in the weeds" a little.   I'll do my best to keep it entertaining.

Questions about Upstream and RESO standards can be grouped into four topic areas: standard names, RETS layout, business rules and the Web API.  In all cases, the Upstream team has a strong, contributing relationship with RESO and we embrace standards whether they are real estate, programming, or other.  Our accelerated use cases (practice) sometimes push the edges of the standards as they are currently defined.  Our commitment was, and remains, to adapt with feedback and contribute those learnings in the hope to contribute what we learn and help to expand the standard.  Now, get out your machete… we're going into the weeds.

Standard Names
 
All listing input, listing data storage, and listing delivery in Upstream is natively implemented using RESO standard names from the Data Dictionary.  Upstream and RPR implemented its initial database schema and API endpoints (including collections and field names) initially using RESO Data Dictionary 1.4 and later updated to match version 1.5 in the fall of 2016.
 
Our pilot only extends to five MLSs, which include fields that are not currently the Data Dictionary.  In those cases, we use a naming convention and schema that can be changed as the Dictionary evolves.  Additionally, Upstream is parcel-centric vs. listing-centric.  Updating listings from a parcel-centric database introduces new dimensions to listing characteristics and enumerations from assessment and other sources of property data, and business rules.  As we discover and document these nuances, we contribute our findings to accelerate the knowledge base for expanding the Dictionary.
 
So why are there questions about standard names?   Why can't Upstream pass field level compliance?  It's a case for… well, case.   The RESO Certification requirements state:  

"The applicant’s field name  MUST  identically match the Data Dictionary Standard Name when the definitions match."

RESO standard names leverage XML-world, PascalCasing conventions.  My intuition is the original RETS layout lived in an XML world.  However, the development community, and RESO, has evolved and adopted the JSON format.  It is now the most common data format used for asynchronous browser/server communication.  Upstream fully embraces JSON with Google's JSON Style Guide which states names must be camelCased, ASCII strings.  Simply put, Iphone becomes iPhone."  Same letters, different casing.   RESO has fully adopted JSON as a data format.  Upstream continued and adopted the JSON naming convention hoping to "skate where the puck is going."  Unfortunately, it's not an identical match which translates, on a technical level, to a failure of field level compliance.
 
RETS Layout
 
The Upstream/RPR team initially proposed a delivery layout that modified the standard RETS format used for listing distribution.  We supplemented it with enumerated collections to make it easier for developers to access enumerations (status variants, historical updates, etc.) in a single pass, and that also better expresses the parcel-centric design (and data richness) of the data architecture. 
 
While some developers appreciated the extension, some argued that the classic RETS format is more familiar and universally accepted.  Well… that's good feedback, so Upstream will offer a flat, RETS layout as an alternative option, using the RESO Dictionary Platinum format.  My intuition is the format will eventually uncover issues once the Web API paradigm shifts from "syndicating" listings to "updating" listings (see Web API).  When we get there, we'll cross that bridge together.
 
Business Rules
 
Many believe that incorporating the business rules across hundreds of MLSs is simply impossible.   And we've discovered that each MLS has hundreds if not thousands of rules. The good news, we've already found patterns.  Although the variables change, the thematic rules are often similar.   We document every rule we discover and hope to publish anonymous, best practices from around the industry.  With that said, RPR and Upstream continue to contribute to the evolution of a business rules standard in the RESO R&D workgroup.  We're excited that RESO has created an ecosystem that can distil broad industry input and create a framework for efficiency while allowing local nuances.
 
Web API
 
As mentioned earlier, the RETS layout and the current Web API definition works well germane to MLS fields and when focused on syndication (distribution).  Similar to Upstream's goals, RESO and the Data Dictionary are expanding beyond MLS fields (as evidenced by the Contact standard) and evolving from simple syndication to “Update.”   Updating presents interesting challenges not seen in today's distribution paradigm.   Let's look at mapping into the MLS vs. mapping from the MLS.  
 
MLSs may have many variants of statuses that get mapped down in a RETS feed for aggregators.  For listing distribution, this is easy:

​But what happens when starting from listing input using the RESO status like Upstream does, and going the other way?  How does the MLS decide?

​And then what happens when creating your listing in two MLSs?

Designating a listing in a RESO status alone may not be enough – other attributes will be needed to indicate the intended status in the MLS.  Similar challenges must be resolved in enumerations: 

• A water source of “Private” in Upstream listing entry using the Dictionary could be “Private Co-op” or “Private/Mutual” in one of our pilot MLSs
•  A water source of “Public” could be “City Water” or “MUD Water.”

 
Upstream creates the need to go faster down the path towards "Update" that RESO was already heading.  We hope that our initiative creates an excellent feedback loop into RESO as we work through these questions with our pilots and subsequent expansion.  
 
Whether it's board participation by various Upstream Board of Managers or workgroup participation by Upstream's partner RPR, we see this as a partnership.  So, if you still plan on storming the castle, put the machete down and bring good wine.   We have long, interesting conversations about the future ahead of us.