Impossibly slow metadata downloads

asked 2022-06-15 11:37:16 -0500

It regularly takes hours to download metadata from Paragon servers. This is an issue across all of our Paragon boards. Most recently, yesterday I began two metadata requests via RETS from https://iamls-rets.paragonrels.com/re... with username RETSiHomeFinder at about 4pm MDT. One request was for full metadata of the OpenHouse resource, and the other for only the SF_1 class of the LivestreamOpenHouse resource. At around 9pm MDT I had to kill both requests as they were still running and data had not yet been returned.

The above is only the latest of what is a standard occurrence. Paragon metadata files are generally far larger than those of other data vendors, largely because of the inclusion of the multiple telephone contact related fields, but even my single-class OpenHouse metadata request could not complete in five hours. Is there anything that can be done to make metadata request download times more manageable. As it stands, we can't be certain that a full work day will be enough time to retrieve metadata, and this is problematic. We do not experience any similar delay in actual data downloads.

Thank you!

edit retag flag offensive close merge delete

Comments

Hello!

We are currently investigating why the metadata is large, as soon as we have an update I will reply here!

Thank you,

Sadie gravatar imageSadie ( 2022-06-15 16:43:29 -0500 )edit

Can you include the metadata transaction you are sending to pull the metadata?
Also have you tried the same transaction in a browser to see how long it takes there?
I was able to pull the full set of metadata in IAMLS in under a minute.
Are you sure that there wasn't an error returned that wasn't handled and your code hung?

bwolven gravatar imagebwolven ( 2022-06-17 17:16:08 -0500 )edit

Are you still having issues with this?

bwolven gravatar imagebwolven ( 2022-06-23 08:21:48 -0500 )edit

Yes, still the same. I am unable to login to https://iamls-rets.paragonrels.com/re... via web browser. The server returns "ReplyCode="20036" ReplyText="Miscellaneous server login error. The RETS-Version request header field is malformed" Our RETS client is able to connect to the URL above and is currently attempting a metadata download. Most of the content so far is encoded, and our request is broken up by the debug output. We will review once the download completes, if it does. Please let us know if you see anything on your end.

iHomefinder gravatar imageiHomefinder ( 2022-06-24 11:24:50 -0500 )edit

The RETS Version you were passing was missing part of it.
Also if you send other requests in the browser, make sure to include the rets-version parameter too.

https://iamls-rets.paragonrels.com/rets/fnisrets.aspx/IAMLS/login?rets-version=RETS/1.7.2
bwolven gravatar imagebwolven ( 2022-06-24 17:42:24 -0500 )edit

Thanks for that. I've corrected the loginURL to https://iamls-rets.paragonrels.com/re... and kicked off a metadata request about 15 minutes ago - it is still processing. FWIW, rets version format is correct on most all of the URLs we use, and still this problem has existed across boards for at least 4 years. It may be an issue with our RETS client; if you see anything problematic in your logs, please let us know. Out client performs without issue on all but Paragon boards, and we have as yet been unable to determine the cause. Thanks again.

iHomefinder gravatar imageiHomefinder ( 2022-06-27 11:54:56 -0500 )edit

What are you using for your clent?

bwolven gravatar imagebwolven ( 2022-06-27 12:03:25 -0500 )edit

A proprietary, house-built program.

iHomefinder gravatar imageiHomefinder ( 2022-06-27 12:17:38 -0500 )edit