Impossibly slow metadata downloads [closed]

It regularly takes hours to download metadata from Paragon servers. This is an issue across all of our Paragon boards. Most recently, yesterday I began two metadata requests via RETS from with username RETSiHomeFinder at about 4pm MDT. One request was for full metadata of the OpenHouse resource, and the other for only the SF_1 class of the LivestreamOpenHouse resource. At around 9pm MDT I had to kill both requests as they were still running and data had not yet been returned.

The above is only the latest of what is a standard occurrence. Paragon metadata files are generally far larger than those of other data vendors, largely because of the inclusion of the multiple telephone contact related fields, but even my single-class OpenHouse metadata request could not complete in five hours. Is there anything that can be done to make metadata request download times more manageable. As it stands, we can't be certain that a full work day will be enough time to retrieve metadata, and this is problematic. We do not experience any similar delay in actual data downloads.

Thank you!

iHomefinder's avatar
asked 2022-06-15 11:37:16 -0500
edit flag offensive 0 remove flag reopen merge delete

Closed for the following reason the question is answered, right answer was accepted by iHomefinder
close date 2023-02-17 10:43:22.644573



We are currently investigating why the metadata is large, as soon as we have an update I will reply here!

Thank you,

Sadie's avatar Sadie (2022-06-15 16:43:29 -0500) edit

Can you include the metadata transaction you are sending to pull the metadata?
Also have you tried the same transaction in a browser to see how long it takes there?
I was able to pull the full set of metadata in IAMLS in under a minute.
Are you sure that there wasn't an error returned that wasn't handled and your code hung?

bwolven's avatar bwolven (2022-06-17 17:16:08 -0500) edit

Are you still having issues with this?

bwolven's avatar bwolven (2022-06-23 08:21:48 -0500) edit

Yes, still the same. I am unable to login to via web browser. The server returns "ReplyCode="20036" ReplyText="Miscellaneous server login error. The RETS-Version request header field is malformed" Our RETS client is able to connect to the URL above and is currently attempting a metadata download. Most of the content so far is encoded, and our request is broken up by the debug output. We will review once the download completes, if it does. Please let us know if you see anything on your end.

iHomefinder's avatar iHomefinder (2022-06-24 11:24:50 -0500) edit

The RETS Version you were passing was missing part of it.
Also if you send other requests in the browser, make sure to include the rets-version parameter too.
bwolven's avatar bwolven (2022-06-24 17:42:24 -0500) edit

Thanks for that. I've corrected the loginURL to and kicked off a metadata request about 15 minutes ago - it is still processing. FWIW, rets version format is correct on most all of the URLs we use, and still this problem has existed across boards for at least 4 years. It may be an issue with our RETS client; if you see anything problematic in your logs, please let us know. Out client performs without issue on all but Paragon boards, and we have as yet been unable to determine the cause. Thanks again.

iHomefinder's avatar iHomefinder (2022-06-27 11:54:56 -0500) edit

What are you using for your clent?

bwolven's avatar bwolven (2022-06-27 12:03:25 -0500) edit

A proprietary, house-built program.

iHomefinder's avatar iHomefinder (2022-06-27 12:17:38 -0500) edit
add a comment see more comments