First time here? Check out the FAQ!

Slow Keyset Fetch in JACKSONMLS

  • retag add tags

In our daily reconciliation process to purge/delete old listings, we are observing that the data feed takes several hours (> over 3 hours) to pull a full keyset from the RETS system, especially in the largest property class (residential). We are specifying a pagination size of 5000 listings per fetch. However, we observed that the feed only returns 500 at a time. This max size was also confirmed in our email followup with the MLS.

Our team actually observed and posted on a similar issue as the one mentioned here. We are following best practices as outlined in this post: using Offset and Limit in our pagination, selecting only fields in the InKeyIndex list (L_StatusCatID, L_UpdateDate, and L_ListingID), etc. Can you confirm that these fields are truly in the InKeyIndex? And recommend how the response time can be improved? We'd also like to know if the pagination size can be increased, as the current limit of 500 records means a little over > 3000 round trips to fetch all listings from the residential class. Conversely, in our other Paragon MLS feeds, we are able to fetch/paginate full keysets in a very timely manner.

Our RETS url is:

ChristineBeta's avatar
asked 2020-11-09 14:29:49 -0500
edit flag offensive 0 remove flag close merge delete


Can you provide me with the account name (NO PASSWORDS).

bwolven's avatar bwolven (2020-11-09 18:11:55 -0500) edit

Yes, it's: REALSCOUTrets


ChristineBeta's avatar ChristineBeta (2020-11-09 19:19:07 -0500) edit
add a comment see more comments

1 Answer


The InKeyIndex functionality is only enabled when Limit=NONE.
I would suggest trying to do the pull of those three fields with it set to NONE.
You may need to increase your transaction timeout value to ensure it completes.
When I pulled all for Property:RE_1 it took around 3 minutes to download all 1,609,707 records.
Using query ((L_UpdateDate=2020-11-02T22:00:00Z-2020-11-09T21:00:00Z)|(L_Last_Photo_updt=2020-11-02T22:00:00Z-2020-11-09T21:00:00Z)) it took around 3 seconds to return 8,286 records.

bwolven's avatar
answered 2020-11-10 08:26:50 -0500
edit flag offensive 0 remove flag delete link


Thanks for clarifying. You're right, I just double-checked the documention re: the use of InKeyIndex + Limit=None. But to clarify, we would like to use pagination if at all possible for the jacksonmls feed (as opposed to fetching all at once), as this is how we process other Paragon MLSs. Other feeds are able to return the entire keysets within 20 - 30 minutes. Since the Limit + Offset is needed to paginate, can you suggest if the 500 max limit can be increased? OR what else can be done to query/paginate the data set in a timely manner?

ChristineBeta's avatar ChristineBeta (2020-11-10 11:56:39 -0500) edit

Hi Bruce, just bumping this in case in got missed... any thoughts to my last comment? We'd prefer to use pagination if at all possible. Thanks.

ChristineBeta's avatar ChristineBeta (2020-11-11 15:23:22 -0500) edit

Checking into it.

bwolven's avatar bwolven (2020-11-11 15:44:55 -0500) edit
add a comment see more comments

Your Answer

Login/Signup to Answer