Recommended approach for purging listings that are no longer available

removed
listings
retag add tags

Hello--

I'm looking for some advice on best practices as it relates to purging listings no longer available to us at the server. Typically this occurs when our client's IDX feed only entitles them to specific listing statuses (Active only, Active+Sold only, etc) - when listings switch to a status not available via the client's credentials, from our perspective, they effectively "disappear" from the feed.

This topic has been addressed by other vendors; example from FlexMLS's documentation below (see Step 3): https://www.flexmls.com/developers/re...

I would like to know your recommended approach, to ensure we stay compliant with the MLSs while also minimizing impact on the RETS server.

Please let me know if you need any additional information. Many thanks in advance for your assistance!

IHRTechServices

asked 2019-03-26 16:42:59 -0500

edit flag offensive 0 remove flag close merge delete

Comments

add a comment see more comments

2 Answers

Sort by

oldest

newest

most voted

This would be similar to what they suggested in that link.
Paragon RETS does support InKeyIndex.

Once a day, request the keyfield value from the resource setting Limit=NONE. Make sure your HTTP request timeout is set to a high value because it can take a while sometimes to return the full set.
Alternatively, you can use the Offset, Limit functionality to return the data in batches until you pull the full set.
Use the list of keyfield values returned to determine which ones are no longer valid for your account.

Note: When you do this make sure to check the error responses to keep from wiping data that is still valid but wasn't included because of an error retrieving the list.
Basically, add some sanity checks to your purging process.

bwolven

answered 2019-03-26 16:58:48 -0500

edit flag offensive 0 remove flag delete link

Comments

Thank you! This is very helpful. I will reach out if we have any additional questions.

IHRTechServices (2019-03-27 15:04:10 -0500) edit

Actually - one question - do all of Paragon RETS servers support InKeyIndex? Or does it vary depending on the client? We curate multiple MLSs' feeds data so it would be helpful to know from an efficiency standpoint if all Paragon servers can be treated the same. Thanks!

IHRTechServices (2019-03-27 15:08:49 -0500) edit

Yes all Paragon RETS servers do support InKeyIndex as I noted above.

bwolven (2019-03-27 15:11:01 -0500) edit

Cool, just wanted to be absolutely sure. Thank you!

IHRTechServices (2019-03-27 15:36:05 -0500) edit

@bwolven , are there plans to include Deleted/Expired/OffMarket listings in a way that would allow us to easily delete listings in near real-time on our end? Maybe just a ListingID and Timestamp. Sometimes we see a completely separate resource called "Deleted" which allows us to sync the deletes without reading the entire set of listings from RETS.

usmidx (2020-04-17 11:01:22 -0500) edit

There is a deleted resource that the MLS may enable if you request it.
But that is only a small part of the issue. Most RETS Feeds have filters applied so a listing can still be in Paragon, but not allowed for particular feeds. This may contain status,VOW, IDX or other criteria and it can vary from feed to feed. You may not be able to access the listing but it hasn't been purged or deleted. It is just no longer accessible in your particular feed.The most reliable way is to pull the IDs once a day to verify they are still valid for your feed. There isn't really an easy solution for this.

bwolven (2020-04-17 11:21:30 -0500) edit

add a comment see more comments

I'm finding issues with time and size of results when requesting the listing ids with no limit. We get back 1.4 million records.

I'm currently grabbing a large batch from our database within a date range on the UpdateDate. Works pretty good, but seems like a waste when we get the listings in our database that are old and sold.

My question: once a listing is sold, or completed, will it ever be deleted? I have access to the MLS and I can see that I cannot edit a listing once it has sold, including adding/removing pictures. Knowing this will allow me to adjust my purge query to only check listing that are not sold.

Seems obvious, but I assume that even though an agent cannot delete a listing after it is sold, there might be a situation where the listing is removed by the board for some reason.

unomateo

answered 2020-04-06 13:12:46 -0500

edit flag offensive 0 remove flag delete link

Comments

The MLS business rules determine how long historical information is ketp.
You could just add something like this to the Query used to pull Listing IDs:
((L_StatusCatID=1,2)|((L_StatusCatID=3),(L_ClosingDate=2016-01-01+)))
Adjusting the date in the query to however many years of sold listings you want to retain.
That would limit the how far back the sold listings would be returned.
You can add the same filter to your incremental update scripts too.

bwolven (2020-04-07 08:36:49 -0500) edit

add a comment see more comments

Recommended approach for purging listings that are no longer available edit

Comments

2 Answers

Comments

Comments